Programming log

SPE versions

2012-02-05T12:23:00.001+01:00

If you use SPE version 0.8.4 instead of version 0.8.3 none of the earlier mentioned problems occur. It is a bit strange that if you go to the sourceforge download site for SPE you get 0.8.3 as the recommended version. If you download from the archive link on the SPE blog you can download version 0.8.4.

spe python(x,y) wxPython2.8 problems

2012-02-04T22:32:00.000+01:00

On Ubuntu the nice python editor from Stani (SPE) has started to behave strange since a recent Ubuntu upgrade. When scrolling with the wheel on the mouse lines get stuck. Moves with the cursor keys correct this but it is really annoying. Therefore I now wanted to run SPE on windows with python(x,y). This proved to be a bit more troublesome than expected. After putting the _spe directory obtained from the SPE download on sourcefourge in the site-packages directory of python (\Python27\Lib\site-packages) simply running the SPE.py script did not work. After running as python SPE.py --debug from a command prompt I had the same problem as reported here (an error in Throbber.__init__). Luckily the solution reported on that same site also fixed the problem:

Moving initialization of _filename, _running, and _position to before the call to GIFAnimationCtrl.__init__ resolves the problem.

Interestingly on openSUSE I experienced the same problems. But there I also had to edit smdi.py (_spe\sm\wxp\smdi.py). In the function setTitle of the class Child(Framework) I commented the calls to self.SetTitle (lines 988-991). Calling self.setTitle instead of self.SetTitle (mark the capital) does not fix the problem (you get a maximum recursion limit error instead). The commenting of the code does not seem to affect the execution of SPE. The only remaining problem on openSUSE with SPE is now that I can only run it as root.

cython odeint example

2011-12-10T14:18:00.001+01:00

In response to the request by Ryan I now post a full example of using cython and odeint together. This is a specially written example because the original work contains too much code from my work that I do not want to post on the web. The equations used in the example have been chosen rather randomly.

In this example I have created a sample Model class which is a callable class (you can use an instance of the class as the function parameter for odeint). The Model class is defined in model_cython.pyx and contains all the Cython code. In ode_script.py the model function is integrated with odeint and thus shows how to call the Cython class from a normal python script. Setup.py is needed to compile model_cython.pyx (see this post).

model_cython.pyx:

import sys

#define some constants a c floats but still callable from python scripts
#not really used here but left in here as an example
cpdef float R_GAS,K_B
R_GAS=8.31      #Gas constant J/molK
K_B=1.38066e-23 #Boltzmann constant J/K

#import some often used math functions from the c math library
cdef extern from "math.h":
    double sin(double)

cdef extern from "math.h":
    double fabs(double)

cdef extern from "math.h":
    double exp(double)

#define a function callable from c and python
cpdef float Velocity(float t,float amplitude=1.0,float period=4.):
    cdef f1
    f1=fabs(sin(t*period))*amplitude
    return(f1)

class Model:
    """
    This class can be used to integrate a differential equation
    """
    def __init__(self,float a,float p):
        """
        -a is the amplitude of the sine function that describes the 
            speed oscillation as a function time
        -p controls the period of the sine function
        """
        self.a=a
        self.p=p
        
    
    def __call__(self,float y,float t):
        """
        y is the current state of the initial value problem:
        y: position
        t: time
        """
        cdef float pos
        pos=y
        #Workhardening
        v=Velocity(t,amplitude=self.a,period=self.p)*exp(-pos)
        return(v)

ode_script.py:

#!/bin/env/python

from numpy import r_
import pylab,scipy.integrate,sys
sys.path.append(".")
from model_cython import *


#ampltiude and period
amp=1.0
period=4.0
#set initial position
pos0=0.0
#create instance of Model object with speed amplitude of 1
myModel=Model(amp,period)
#create time vector (4 seconds in 500 steps)
tvec=r_[0.0:4.0:500j]
#integrate ov
y=(scipy.integrate.odeint(myModel,pos0,tvec))
    

#plot results
pylab.figure(1)
pylab.clf()
pylab.subplot(211)
pylab.plot(tvec,y)
pylab.xlabel("Time (s)")
pylab.ylabel("Position (m)")
pylab.grid(b=True)
pylab.subplot(212)
v_list=[]
for t in tvec:
    v_list.append(Velocity(t,amplitude=amp,period=period))
pylab.plot(tvec,v_list)
pylab.xlabel("Time (s)")
pylab.ylabel("Undamped velocity (m/s)")

pylab.grid(b=True)
pylab.show()

setup.py:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

ext_modules = [Extension("model_cython",["model_cython.pyx"],\
    libraries=["m"])]

setup(
    name= 'Generic model class',
    cmdclass = {'build_ext': build_ext},
    ext_modules = ext_modules
)

Hope this helps.

Ubuntu 11 samba mounting problems during booting

2011-11-13T15:42:00.001+01:00

After upgrading to Ubuntu 11 I started to receive errors during booting mentioning CIFS-VFS. A quick search on the internet did not reveal a simple solution. Now I had a more thorough look. The CIFS-VFS error apparently has something to do with the automatic mounting of a samba share on a server in my network. I now know why the error pops up twice. From posts on the web I found that the most likely reason that I get this error is because the network is not up before the mounting is tried. Which makes sense when you get the error

mount error(101): Network is unreachable

(look in /var/log/boot.log)

Ubuntu runs the boot scripts in /etc/init. Here I found mountall.conf and mountall-net.conf. Here I found that the order of the scripts is determined by a line in the scripts in the form

start on runlevel [2345] stop on runlevel [!2345]

When I looked in /etc/init/mountall-net I found the line:

start on net-device-up

which suggests that the script waits for the network to be up before the script is run. The other script mountall.conf has the line "start on startup" in it which I think means that this script is called more or less at the beginning when the network indeed may not yet be up. When the network is up (or supposed to be up) mountall-net is called. This explains why I get the mount error twice during booting. While searching the web for "start on net-device-up" I somewhere found the variant

start on (net-device-up IFACE=eth0):

And this appears to make the problem go away (so now I only have one mnt error left during booting (from mountall.conf) but my samba share does get mounted automatically during booting).

working with openmp and gcc vectorization output

2011-10-14T21:47:00.001+02:00

When working with openmp for parallelisation you can set the number of threads in a bash shell (cygwin) by

export OMP_NUM_THREADS=4

this can be good to do if you do not want to use all "cores" detected automatically on a cpu with hyperthreading.

To see which loops have been automatically vectorized by gcc you have to add the compiler switch

-ftree-vectorizer-verbose=5

Other verbose levels can also be used but for me level 5 seems to give the most relevant information.

march=native

2011-09-18T10:57:00.000+02:00

On the web I found an easy way to find what the gcc compiler option -march=native translates into (can't remember where I found this, it was somewhere on a gentoo forum):

echo "" | gcc -march=native -v -E - 2>&1 | grep cc1

On my Q6600 with gcc 4.4.3 this results in:

/usr/lib/gcc/i486-linux-gnu/4.4.3/cc1 -E -quiet -v - -D_FORTIFY_SOURCE=2 -march=core2 -mcx16 -msahf --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=4096 -mtune=core2 -fstack-protector

Surprisingly on my laptop with an i7-2630QM and (cygwin) gcc 4.5.3 I get:

/usr/lib/gcc/i686-pc-cygwin/4.5.3/cc1.exe -E -quiet -v -D__CYGWIN32__ -D__CYGWIN__ -Dunix -D__unix__ -D__unix -idirafter /usr/lib/gcc/i686-pc-cygwin/4.5.3/../../../../include/w32api -idirafter /usr/lib/gcc/i686-pc-cygwin/4.5.3/../../../../i686-pc-cygwin/lib/../../include/w32api - -march=core2 -mcx16 -msahf -mpclmul -mpopcnt -mavx --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=generic

I was expecting a -march=corei7-avx but I guess that is only from gcc 4.6?

Using Cython

2011-07-24T18:33:00.000+02:00

The easiest way to convert old python code for use with cython is to rename your python file into something with the extension .pyx. This .pyx file can then be compiled with the setup.py (for distutils)script.

Here is an example setup.py which also shows how to include the math libray (-lm for gcc)

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

ext_modules = [Extension("BvL_cython",["BvL_cython.pyx"],\
    libraries=["m"])]

setup(
    name= 'BvL model class',
    cmdclass = {'build_ext': build_ext},
    ext_modules = ext_modules
)

The reason that I was interested in Cython was the long calculation times I encountered while doing a multi-variable optimization with a function evaluation that involved solving a differential equation with scipy.integrate.odeint. By simply replacing the class that contained the differential equation with a Cython version the calculation time dropped by a factor 5. Not bad for half a Sunday afternoons work.

First steps with Cython and Python(x,y)

2011-07-24T16:38:00.000+02:00

Started today experimenting with Cython. With Cython you can write normal python code and compile it to C code. When you start to insert type information the speed up you get compared to normal python can be quite dramatic. The compiled c-code can be called from python (seemingly as if it was normal python code your calling). It is all nicely introduced here: Cython tutorial

On windows I am using Python(x,y). When you install it you have to tell the installer to include Cython. As c-compiler the Python (x,y) package includes mingw. Unfortunately Cython by default seems to expect a Visual C compiler. So the only way (so far) I found to use Cython is to run python from a normal command prompt when using the setup.py method (distutils) as described in the Cython Tutorial as follows:

python setup.py build_ext --inplace --compiler=mingw32

otherwise you get this error message:

error: Unable to find vcvarsall.bat

compiler options in QtCreator

2010-11-22T21:10:00.000+01:00

Selecting the release build over the default debug build will enable the -O2 compiler option but under Projects settings you can only add make options. To add more compiler options such as -march=core2 you can add a line to your project file:

QMAKE_CXXFLAGS += -march=core2

Working with a second y-axis in matplotlib

2010-11-10T21:57:00.002+01:00

Recently I had to create a plot with data on a second y-axis in matplotlib (python). This is actually more involved than I expected. After googling around I put this together

First you store the axes object returned from a subplot(111) command

ax1=subplot(111)

then you plot as usual your data which has to go to the first y-axis. Labels you have to set for the ax1 object:

ax1.set_xlabel(r"Position ($\mu$m)",fontsize=20)
ax1.set_ylabel("Concentration (wt %)",fontsize=20)

The second y-axis is created by creating a twin of your current axes object. The twin is actually not identical but has a mirrored y-axis (i.e. your second y-axis).

ax2=twinx()
plot(....)
ax2.set_label("Other conc (wt%)",fontsize=20)

So far so good. Normally I use the follow function to change the size of text of the ticks

def setTickSize(size=None):
 """Set the tick label size (both x and y axis) to the specified value or 
 default to 'large'
 possible values are 'small', 'medium', 'larger', 'x-large', 'xx-large'
 'x-small', 'xx-small' or integer values
 !!!!!!!!Warning!!!!!!!!!
 Use this function only when you are finished adding curves. If curves
 change the axes ranges the ticks get screwed up (sorry)
 """

 if size is None:  #default large
  size='large'
 elif type(size) == str:
  if size not in ['small', 'medium', 'larger', 'x-large', 'xx-large','x-small', 'xx-small']:
   print "Wrong fontsize specifier given, use:"
   print 'small', 'medium', 'larger', 'x-large', 'xx-large', 'x-small', 'xx-small'
   print 'defaulting to large'
   size='large'
 elif type(size) != int:
  print "Wrong fontsize specifier given, use:"
  print 'small', 'medium', 'larger', 'x-large', 'xx-large', 'x-small', 'xx-small'
  print 'or an integer value'
  print 'defaulting to large '
  size='large'
 
 tmp=pylab.xticks()
 pylab.xticks(tmp[0],[str(int(x)) for x in tmp[0]], size=size)
 tmp=pylab.yticks()
 pylab.yticks(tmp[0],[str(int(x)) for x in tmp[0]], size=size)

You have to change the type cast int(x) into something else if you want floating point values on your axes. However, it turns out that it is near impossible to do this for the ticks on the second y-axis. After some more googling I found a much easier alternative. Matplotlib can read settings from a user-preferences file somewhere in the user-home directory. Somehow I can never find this file (I use a lot of different windows/linux/python combinations). But from a script you can also set these preferences! Which of course then only apply to the remainder of the script:

import matplotlib as mpl
mpl.rcParams['xtick.labelsize']=20
mpl.rcParams['ytick.labelsize']=20

The nice part is that the labelsize applies to both your first and second y-axis. You can find more info on the matplotlibrc file on the sourceforge matplotlib site

The only issue I could not solve is to create one legend for the curves on both axes. If you legend() you only get the legend for the current active axes. On windows calling it again just replaced the other legend on linux they were both displayed on top of eachother (which gives you the option to use e.g. legend(loc=2) for the left y-axis and legend(loc=1) for right y-axis).

Change legend line width in Qwt

2010-09-26T13:39:00.004+02:00

In the past I made a small program that basically just plots a number of columns from a text file. This program is written using the Qwt plotting library. The program can simultaneously plot the columns from multiple text files. The different columns in a file are given different colours and for columns from different files a different line style is used (normal, dashed, dotted or dash-dot). A legend showing which line is which is also displayed. Here I had the problem that with the default legend properties the width of the line shown in the legend is so small that you can not distinguish between a normal and a dashed or dashed dotted line. After some searching on the web I found a good description of how to change the width of the line in the legend. If you follow the link you find that they call the line section displayed the legend icon (that is why it took me some time to find that website). In the end I modified their code a little bit. My program was using plain QwtPlotCurve variables. So I made a derived class FractionCurve which implements the updateLegend function. The total code of this very simple custom Curve class is given below. Because my program uses the QwtPlotCurve constructor that takes a QString as an argument I also had to implement that constructor for the derived FractionCurve class.

#include <qwt_legend.h>

#include <qwt_legend_item.h>


class FractionCurve : public QwtPlotCurve

{

public:

void updateLegend(QwtLegend *legend) const;

FractionCurve(const QString);

};



FractionCurve::FractionCurve(const QString s) : QwtPlotCurve(s)

{

;

}



void FractionCurve::updateLegend(QwtLegend *legend) const

{

QwtPlotCurve::updateLegend(legend);

QwtLegendItem *lgdItem =

dynamic_cast<QwtLegendItem*>(legend->find( this ));

if ( lgdItem )

lgdItem->setIdentfierWidth(24);

}

And an example of a plot with the new legend:

Nice syntax highlighting in Blogger

2010-08-14T12:53:00.000+02:00

There is an easy way to make source code snippets look great on your blog. You have to change your template once and after that you can just surround your code with a simple tag. One thing you must not forget to do is to "escape" your source code so it only contains valid html code. Manually this means replacing any less than sign with appropriate html code.

Everything is nicely explained here:

syntax highlighting tutorial

and here is a direct link to a site to escape your html

quick escape

Reading a vtk file with multiple scalar fields

2010-08-08T18:43:00.004+02:00

The Micress phase simulation package can store data files in .vtk format. It stores multiple scalar values in a single file, where the scalar are stored as structured points. For example I received I file containing concC, phas and rex. I would like to read in these files and convert them to something else. So the main goal was to obtain arrays with the 3 scalar fields. The code is not yet complete but the reading is quite involved and it took me a lot of time to patch the required information together from all different examples and the DOxygen online manual of VTK (which I have come to dislike to a great extend). Perhaps the VTK books would have helped me out here but I am not willing to buy them just for this single application of VTK.

Okay here we go:

//First you have to create a reader object:
vtkSmartPointer < vtkstructuredpointsreader> reader = vtkSmartPointer < vtkstructuredpointsreader>::New();
reader->SetFileName(inputFilename.c_str());
//Now we can get the number of scalars in the file
vtkIdType numScal=reader->GetNumberOfScalarsInFile();
std::cout << "number of scalars in file: " << numScal << std::endl;
//Show which scalars we have
int i;
for(i=0;i< numScalComp;i++){
  std::cout << "Scalar "<< i <<": " << reader->GetScalarsNameInFile(i)<< std::endl;
}
//To read a certain scalar field we have to tell the reader which one we want
//we do that by name in this example we take the third one
reader->SetScalarsName(reader->GetScalarsNameInFile(2));
reader->Update();  //I think this actually makes the reader do something
//the rest is what vtk people call setting up the pipeline
//To get to the point data we need a few intermediate steps and objects
vtkStructuredPoints* structuredPoints = reader->GetOutput();
vtkPointData *pd=structuredPoints->GetPointData();
pd->Update();
vtkDataArray *scalars ;
scalars=pd->GetScalars(reader->GetScalarsNameInFile(2));
//here the same name as selected in the reader must be used!!
//I tried this line with the SetScalarsName call to the reader and
//that doesn't work.
//Now we can access the points for example as
vtkIdType numPoints = structuredPoints->GetNumberOfPoints();
for(i=0;i< numPoints;i++){
   std::cout<< i<< ": " << scalars->GetComponent(i,0)<< std::endl;
}

The next step is to write a .vtk file from my own data. The mayavi2 and ParaView programs look quite nice for imaging of 3d data.

Weird experience with Gmail

2010-03-28T14:10:00.000+02:00

Today I tried to login to my gmail account using chrome under Ubuntu (I always use https). After trying for a period of 15 minutes during which it kept telling me that the website was unavailable I was about to ask my wife if she alaso had problems reaching her gmail. Then I tried firefox and I could login immediately. I tried Chrome once more and still could not login. Now I can use chrome to login to gmail as well.

Weird

Problems with Qt and Path environment variable

2009-12-28T09:50:00.000+01:00

Qt Creator kept giving strange error messages. Message such as "mingw32-make" not found in the path. Only after removing all references to my cygwin installation in the windows (XP) PATH environment settings does Qt Creator work again.

Cygwin (from a bash shell) still seems to work after removing these path entries.

Path for qt

2009-12-22T16:43:00.000+01:00

It turned out that my cygwin installation also contains some qt elements. They interfered with the Qwt examples. After moving

D:\Qt\2009.05\Qt\bin

to the front of the path I can now directly run the Qwt examples by double clicking in an explorer window.

By the way in

D:\Qt\2009.05\bin

the dll's needed to run Qt Creator are stored and not the dll's you build your own software against. With Qwt it is possible to build an plugin for Qt designer. However this plugin when build with the free Qt package does not work because Qt Designer was build against these other dll's. Not nice.

Python script for lame mp3 encoding

2009-12-22T09:35:00.000+01:00

For some reason EAC refused to encode a CD to mp3. Therefore I copied the .wav files and downloaded the lame package from sourceforge. Lame was then compiled and installed under cygwin (1. ./configure 2. make 3. make install). Then with this small python script all tracks could be compressed to mp3:

import os

for i in range(1,28):
call_text="lame -V2 Track%02d.wav Track%02d.mp3"%(i,i)
print call_text
os.system(call_text)

(on the web I had to look up how to do the os.system call and how to pad with zeros to the numbering of the tracks right 01, 02 ,03

qHash problem

2009-12-21T22:48:00.001+01:00

The qHash problem was also found by other people

according to
this forum entry

you can just remove the QWT_TEMPLATEDLL section (I did this by changing the line
#if defined(QWT_TEMPLATEDLL)
near the

template class QWT_EXPORT QList;

line
into

#if defined(QWT_TEMPLATEDLLxx)

after this the compile runs fine and I have a fresh qwt5.dll file.

By opening the example.pro file in the Qt Creator and running qmake from the build menu followed by a run I also managed to compile the examples. The examples don't run because qwt5.dll cannot be found. Copying the dll into the examples/bin folder lets you run the examples from Qt Creator. You still cannot run the examples directly from windows because then the qt dll's cannot be found. Running the examples from the command prompt that comes with the nokia Qt installation package works though.

Better option is of course to just add the folders with the qt dll's to the windows environment. I choose to copy the qwt dll into the same folder as the other qt dll's (safes an entry into the path environment setting)

Qwt and Qt (mingw version) next step

2009-12-21T22:18:00.000+01:00

When you search on google for qwt one of the first hits is a download link on sourceforge for version 4.2.0. The most recent version at the moment however is 5.2.0. Which was made for Qt 4.5. The Qt package from Nokia now seems to come with Qt 4.6.

When compiling the Qwt package from the Qt Creator it now comes much further but it runs into an error compiling qwt_valuelist.h:

2009.05/qt/include/QtCore/../../src/corelib/tools/qhash.h:875: error: call of overloaded 'qHash(const double&)' is ambiguous

Starting with Qt and Qtw

2009-12-21T21:49:00.000+01:00

Today I downloaded Qt free version (4.6.0) and installed it on my labtop (windows XP). I also downloaded the Qwt plotting library (4.2.0). After installation Qwt does not compile when I open the qwt project in the Qt Creator. The binary examples also do not run because a dll is missing. I already had this problem once when I installed Qt and Qwt on my desktop machine (running Vista). Of course now I cannot remember what I did then to make it run that time. And that is why I decided to start this blog. Just to make my own notes to help me remember these kind of things.

Maybe it will help someone else too.