using SWIG to connect python and a c++ dll

Although I am a great fan of Cython when you already have a class defined in C++ code you do not want to rewrite it for Cython. Already a long time ago I once used SWIG to create a link between C++ classes and python. This is fairly easy as long as you have clear interface functions which you can use to pass data to and from your class. Nevertheless, it took me some effort to get things running for my new class.

Following the SWIG tutorial I started with something like this myExample.i

%module myExample
 %{
 /* Includes the header in the wrapper code */
 #include "myModule.h"
 #include "myModule2.h"
 %}
 
 /* Parse the header file to generate wrappers */
 #include "myModule.h"
 #include "myModule2.h"

After compiling this I got a module that I could import in Python. However, the module did not contain the classes I had defined in my C++ code. Because there are not so many clear examples of SWIG and C++ I could not really find an explanation for this. Only after recreating the exact example of the tutorial (using copy/paste from the web) I found that I made a simple typing error: the #include in the second part of the myExample.i should be %include!

%module myExample
 %{
 /* Includes the header in the wrapper code */
 #include "myModule.h"
 #include "myModule2.h"
 %}
 
 /* Parse the header file to generate wrappers */
 %include "myModule.h"
 %include "myModule2.h"

Now everything works nicely. Compiling is not very difficult. I have setup a Makefile for this:

CPP=g++

CFLAGS=  -Wunused  -O3

PYTH_INCL = /c/Python27/include
PYTH_LIB = /c/Python27/Libs

PYLIB = -lpython27

OBJS= myModule.o
modOBJS= myModule2.o
SRC_DIR = ..
all: myExample_mod

myExample_mod: $(modOBJS) myExample.i
 swig -I.. -python -c++ -shadow myExample.i
 $(CPP) -I$(PYTH_INCL) $(INCLUDE) -c myExample_wrap.cxx
 $(CPP) -shared -L$(PYTH_LIB) myExample_wrap.o $(OBJS) $(PYLIB) $(LIBS) -o _myExample.pyd

%.o: $(SRC_DIR)/%.cpp $(SRC_DIR)/%.h
 $(CPP) -c $(CFLAGS) $(INCLUDE)  $< -o $@

With this Makefile you can use import myExample in your python code to import the module. Thus far I have only used simple interface methods in my classed that work with integers and floats as input and output variables. This makes communication between the module and python very simple (no special effort seems to be required). I assume that if you want to pass arrays, strings, etc. things will become more difficult.

cython -mno-cygwin problems

The command I use to compile a cython extension on windows:

python setup.py build_ext --inplace --compiler=mingw32

(see also these two posts) has problems with newer mingw/cygwin installations. Already for some to the compiler option -mno-cygwin is no longer used and the mingw compiler should be called directly. When you run the setup.py script with the compiler flag as shown, however, python automatically adds the -mno-cygwin flag. This then stops the compiler. The best solution for this I have found so far is to change the

c:\Python27\Lib\distutils\cygwinccompiler.py

file. Simply remove the -mno-cygwin option everywhere in the definition of the Mingw32CCompiler class (just search for no-cygwin in this file and you will find it).

Looks like I also could have googled this instead of finding out again how to use grep to search in subdirectories which is not as simple as adding the -r option ( find ./* -type f -exec grep -l "no-cygwin" {} \;).

Location drive specification MinGWPortable

A long time ago I found a portable version of MinGW somewhere on the web. It is sometimes very convenient to have a gcc compiler ready to run on a computer that is not your own. However, I used it mainly on one computer and when I finally did try to run it on a different machine it didn't work. The problem was that it was looking for the MinGW and msys stuff on the wrong drive. Although not very portable it is easy to change this as long as you remember that msys emulated a unix environment! Just change the drive in

/etc/fstab

It took me some time to find this. But instead of searching myself I could have looked here.

Somehow I am no longer able to find my original MinWGPortable on the web anymore. That is a pity because the msys terminal is extremely nice: you can resize it in any direction! Perhaps I should have another look here.

Windows application icon in Qt Creator

In an earlier post I described how to add icons to a Qt application using Qt Creator. With the procedure described there, the application shows the icon in the window title bar and on the taskbar. However, when you browse to the folder containing the application in windows, the executable will not be shown with the icon. The same is true when you create a shortcut to the executable (windows will also say that the executable does not contain an icon).

As described here, there is a simple windows specific method to add an application icon that will change the appearance of the executable when browsing under windows (and when creating a shortcut to the executable). You must first create a windows specific resource file which has the extension .rc e.g. myapp.rc. This simple text file then needs to contain the (single) line:

IDI_ICON1 ICON DISCARDABLE "images/myapp.ico"

The final step is then to add this resource file in the Qt project (.pro) file:

RC_FILE = myapp.rc

This works for me when I put the myapp.rc in the same folder as where the Qt project file is located (and that contains the images folder). I did not try any other file locations.

pythonxy cython mingw problems

Again I was having issues using a cython compiled package with pythonxy. For some reason when running from a ipython terminal started from pythonxy, I would get a DLL load error. If I would run the python/cython program directly (i.e. from a dos box: python my_program.py) then it would run fine. By default if you compile a program with mingw you need a dll with the std c or c++ library (typically this dll has a difficult name as libgcc_s_dw2-1.dll). As I have mingw installed in more than one location (32bit, 64bit, QtSDK, Python(x,y)) I think it cannot find the right dll (although I could not detect which dll it was using). The problem can be most easily solved if you link the std library statically. This you can do by simply adding "stdc++" to your libraries field in setup.py.

So my new setup.py looks now something like

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy

ext_modules = [Extension("BvL_cython",["model_cython.pyx"],\
    libraries=["m","stdc++"],include_dirs=[numpy.get_include()])]

setup(
    name= 'Model class',
    cmdclass = {'build_ext': build_ext},
    ext_modules = ext_modules
)

(This also shows how to automatically detect the location of the numpy include directories)

Color text output from python

At the moment I run a minimization routine build in Cython/Python that outputs a lot of text to the terminal. Not all text is equally interesting and I was wondering if it is difficult to output some of the more important messages in a different color. It turns out that this is very easy. On stackoverflow Dietrich Epp posted a nice example which I have changed a little to select the color with a single character string:

def hilite(string, status, bold):
    attr = []
    if(sys.stdout.isatty()):
        if status=='g':
            # green
            attr.append('32')
        elif status=='r':
            # red
            attr.append('31')
        elif status=='y':
            # yellow
            attr.append('33')    
        elif status=='b':
            # blue
            attr.append('34')
        elif status=='m':
            # magenta
            attr.append('35')
        if bold:
            attr.append('1')

        return '\x1b[%sm%s\x1b[0m' % (';'.join(attr), string)
    else:
        return(string)

More info on ANSI escape codes is given on Wikipedia. You can also do things like underline. So far I have only tested this on Linux so I do not know if it also works on Windows.

Update:
This does not work on Windows (windows terminals do not work with ANSI codes). Luckily there is a python package you can use to make it work on windows after all: colorama. Simply call colorama.init()
(after import colorama) and you can use the above code again. However, I noticed that if you use ipython this garbles your prompt (and other things). To work around this I call colorama.init() in the functions that use the ANSI codes in the output. Then when the script is finished you have to call colorama.deinit() and ipython will behave normally.

header file location for numpy arrays (cython)

Today was the first time I tried working with numpy arrays in Cython. I got a compilation error saying that the header file for numpy could not be found. Here I found an easy fix for this problem. The setup.py script can be changed to automatically detect and add the numpy include path:

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy

ext_modules = [Extension("xx_cython",["xx_cython.pyx"],\
    libraries=["m"],include_dirs=[numpy.get_include()])]

setup(
    name= 'xx model class',
    cmdclass = {'build_ext': build_ext},
    ext_modules = ext_modules
)