Minimize button in a QDialog in PyQt

Often it is easy to write a small application in PyQt as a QDialog (instead of using a QMainWindow). If it is intended as an application that has to be open for a longer time (e.g. while the user is using other programs) it is nice if the program can minimize (e.g. when the user clicks the show desktop button in windows (or the nice windows-key+d key shortcut). However, by default the QDialog window only has a close button. Luckily adding minimize and maximize buttons is easy:

from PyQt4.QtCore import *
from PyQt4.QtGui import *


class MainForm(QDialog):

    def __init__(self, fn=None,parent=None):
        super(MainForm, self).__init__(parent,\
           flags=Qt.WindowMinimizeButtonHint|Qt.WindowMaximizeButtonHint)

Other flags can be found here

SQLITE DATETIME troubles

Recently, I have started to play with SQLITE. For the database creation and queries I actually use PyQt but that is not really the issue. I was unaware of the complications with SQL databases and DATETIME formats. I made the wrong assumption that if I provided input that did not give any errors and looked good in the nice SQLite Database Browser everything would be okay. What happened was that I filled a datetime column with strings like '09-11-2014 12:43:00'. This actually is one of the few not supported datetime formats. When running a query where the datetime has to be before or after another (similarly formatted) datetime this actually seemed to produce sensible results. Only when I tried to do more fancy datetime stuff like datetime('09-11-2014 12:43:00','-1 month') did things suddenly stop working. I think sqlite is treating the wrong input for a datetime column as a string and is actually doing a string comparison when comparing two wrongly formatted datetime values (which can sometimes give sensible output). The solution is of course simple: only use one of the supported datetime formats when inserting data into a table.

Cython modules not automatically rebuild through setup.py on changes of pxd files

After some frustrating hours I discovered that if you use a distutils setup.py script to create a cython module the module is not updated if you only change a pxd header file. Especially in a bigger project with many cython modules this can be easily forgotten, so in case of unexpected behaviour just try removing all pyd files so everything will be build fresh.

It is perhaps even saver to touch all .pyx files because if you remove the pyd file the .c file is not necessarily recreated!

interp1d x-axis order must be in ascending order

Today I discovered that the nice scipy 1d interpolation function interp1d requires that the values in x-array passed as the first argument are in ascending order. Perhaps this makes sense but it took me some time to figure out that this was the reason I was getting out-of-bounds errors (my x-array was in descending order).

Simple txt2csv python script

Often I have a text files with columns that are separated by spaces or tabs or a mixture of them. These files can be read by a typical spreadsheet program but this usually requires some extra mouse clicks to tell the spreadsheet program how to interpret the text file. Csv files (comma separated files) are often much better recognized by a spreadsheet program. Therefore I have written a simple python function/script to change a text file to csv file where spaces and tabs are replaced by a separator (default set to semicolon). Multiple spaces are combined into a single separator (which is what I typically want). Leading and trailing spaces on a row are removed. It is amazing how little python code is needed for this (thanks to python's nice string handling). The magic python line csvline=s.join(line.split()) in the script below is based on a very useful item on Stack Overflow.

import sys

def txt2csv(fnIn,fnOut,separator=';'):
    fin=open(fnIn,'r')
    if(not fin):
        print "Error opening %s"%(fnIn)
        sys.exit(-1)
    fout=open(fnOut,'w')
    for line in fin.readlines():
        s=separator
        csvline=s.join(line.split())
        fout.write(csvline+'\n')
    fin.close()
    fout.close()

if __name__== "__main__":
    try:
        fnIn=sys.argv[1]
        fnOut=sys.argv[2]
    except:
        print "Usage: python %s fnIn fnOut"%(sys.argv[0])
        
    txt2csv(fnIn,fnOut)

guiqwt string based function plotter

For almost all plotting from python I use matplotlib, but recently I noticed an interesting alternative called guiqwt. Guiqwt seems to be faster and more tuned towards using plots in a UI environment. In the youtube movie Introduction to Wolfram Language a nice demo is shown where a gui is assigned automatically to a function with sliders for the function parameters so the user can easily explore the function behavior. If you often work with empirical functions to reproduce certain experimentally observed trends it can be very nice to have such a quick visualization of the influence of the chosen function parameters. This gave me the idea to try to make a generic function plotter with python/guiqwt where the user can supply the function by typing a string from which the parameters are then automatically recognized and assigned a slider so the user can explore the function behavior. This turned out to be a little more involved than anticipated but the result is acceptable:

To make life a little easier the user has to write function parameters (and independent variables) starting with an underscore. This saves the effort of trying to recognize all possible mathematical functions from the string (e.g. no searching from tan, sin, exp, sqrt etc. required) which would have been nicer of course. All in all the code is about 400 lines (see below), a bit more than expected. But it was a nice exercise to get familiar with PyQt and guiqwt.

from guidata.qt.QtGui import QWidget, QVBoxLayout, QHBoxLayout,QPalette
from guidata.qt.QtGui import QPushButton,QSlider,QGridLayout,QLabel,QFont
from guidata.qt.QtGui import QCheckBox,QLineEdit,QComboBox,QSizePolicy
from guidata.qt.QtCore import SIGNAL
from guidata.qt.QtGui import QApplication
from guidata.qt.QtCore import Qt
from guiqwt.plot import CurveWidget
from guiqwt.builder import make
from guidata.configtools import get_icon

import numpy
from numpy import *
from functools import partial


class TFunctionParameter:
    """
    This class defines the properties of a function parameter.
    The properties are:
    -Name
    -Value
    -Minimum allowed allowed parameter value
    -Maximum allowed allowed parameter value
    """
    def __init__(self,name,value,minValue=None,maxValue=None,scientific=False):
        """
        Constructor for the TFunctionParameter class
        Arguments:
        name: name of the parameter (this is the string you use in the function definition!)
        value: (initial) value of the parameter
        minValue: if provided the minimum allowed value of this parameter. Otherwise default 0.1*value
        maxValue: if provided the maximum allowed value of this parameter. Otherwise default 10.0*value
        """
        self._name=name
        self._v=value
        self._scientific=scientific
        if(minValue):
            self._minV=minValue
        else:
            self._minV=0.1*self._v
        if(maxValue):
            self._maxV=maxValue
        else:
            self._maxV=10.0*self._v
            
    def Scientific(self):
        return(self._scientific)    
    
    def v(self):
        return(self._v)
    
    def setValue(self,v):
        if(v<self._minV):
            self._v=self._minV
        elif(v>self._maxV):
            self._v=self._maxV
        else:
            self._v=v
    
    def getValue(self):
        return(self._v)
        
    def minValue(self):
        return(self._minV)
        
    def maxValue(self):
        return(self._maxV)
        
    def name(self):
        return(self._name)
        
    def checkV(self):
        """
        Checks if parameter value still between min-max. If not
        value is adjusted to fall in allowed range.
        """
        if(self._v>self._minV):
            self._v=self._maxV
        elif(self._v<self._minV):
            self._v=self._minV
        
    def setMinValue(self,minV):
        """
        Adjust the minimum allowed value for this parameter. The parameter
        value is adjusted to fall in the new range. This function returns
        the current (possibly adjusted) parameter value.
        The new min value is not allowed to be bigger than the current
        maximum allowed value.
        """
        if(minV>self._maxV):
            self._minV=self._maxV
        else:
            self._minV=minV
        self.checkV()
        return(self._v)
        
    def setMaxValue(self,maxV):
        """
        Adjust the maximum allowed value for this parameter. The parameter
        value is adjusted to fall in the new range. This function returns
        the current (possibly adjusted) parameter value.
        The new max value is not allowed to be smaller than the current
        minium allowed value
        """
        if(maxV<self._minV):
            self._maxV=self._minV
        else:
            self._maxV=maxV
        self.checkV()
        return(self._v)
        
    def setMinAndMAxValues(self,minV,maxV):
        """
        Adjust the minimum and maximum allowed values simulataneously.
        Use this function to specify a completely new range for this
        parameter
        
        This function returns
        the current (possibly adjusted) parameter value.
        """
        self._minV=minV
        self._maxV=maxV
        self.checkV()
        return(self._v)
    
class TFunctionUI(QWidget):
    """
    The idea of this class is to provide an automatic user interface
    to adjust all function parameter values and see the effect on the
    function behaviour.
    """
    def __init__(self,parent):
        """Constructor of the TFunctionUI class"""
        self._parameterDict={}
        QWidget.__init__(self, parent)
        self.setMinimumSize(520, 400)
        self.x = [0,1]
        self.xvec=[0,1]
        self.y = [0,1]
        self.func = self.calc
        self.title="f"
        self.autoScale=True
        #---guiqwt curve item attribute:
        self.curve_item = None
        #---
        
    def calc(self):
        """
        Based on function string calculate function
        """
        xpar=self._parameterDict[self._xName]
        xvec=numpy.r_[xpar.minValue():xpar.maxValue():-250j]
        evalString=self.funcString[:]
        for par in self.paramLst:
            if(par==self._xName):
                evalString=evalString.replace(par,"xvec")
            else:
                evalString=evalString.replace(par,"self._parameterDict['%s'].v()"%(par))
        
        self.xvec=xvec        
        f=eval(evalString)
        return(f)
        
    def addParameter(self,p):
        """
        Add a parameter.
        Arguments:
        p - The parameter to add. Must be an instance of TFunctionParameter.
        """
        self._parameterDict[p.name()]=p
        
    def defineX(self,x):
        """
        Define the value on the x-axis. 
        Arguments:
        x - the parameter to plot on x-axis. Must be an a string and part of the parameter list
        """
        self._xName=x
        
    def parValue(self,p):
        """
        Returns the current value for parameter p
        """
        return(self._parameterDict[p.name()].v())
        
    def getParametersAndFunction(self,fs):
        """
        Extract the parameters from the function string
        fs: function string
        """
        math_symbols=['<','>','=','*','+','-','/','(',')']
        #Process string
        #find first occurence of _ to indicate a parameter
        NParams=fs.count('_')
        self.paramLst=[] #Stores parameter names in the order as they appear in the equation
        self.funcString=fs[:]
        cntParams=0
        #~ print self.funcString
        for np in range(NParams):
            index=fs.index('_')
            #Now look for first math symbol to signify end of parameter name
            i=index+1
            while(i<len(fs) and (fs[i] not in math_symbols )):
                i+=1
            #Now the parameter name is from index upto i-1
            parName=fs[index:min(i,len(fs))] #include the underscore
            fs=fs[i:]
            if(parName in self.paramLst):
                continue #Do not count same parameter double
            
            p=TFunctionParameter(parName,1.0,-5.0,5.0,scientific=False)
            self.paramLst.append(parName)
            self.addParameter(p)
        
        
    def slideChange(self,key):
        """
        Process slide change. Key indicates to which parameter
        the slide belongs to
        """
        v=self.slideDict[key].value()
        float_v=v/1000.0
        p=self._parameterDict[key]
        p_value=p.minValue()+(p.maxValue()-p.minValue())*float_v
        p.setValue(p_value)
        if(p.Scientific()):
            self.valDict[key].setText("%5.4e"%(p_value))
        else:
            self.valDict[key].setText("%6.3f"%(p_value))
        self.process_data()
        
    def toggleAutoScale(self,v):
        self.autoScale=self.autoScaleCheckBox.checkState()
        self.process_data()
        
    def parMinChanged(self,key):
        """
        Process a change of a minimum parameter value
        """
        p=self._parameterDict[key]
        p.setMinValue(self.minLEDict[key].text().toFloat()[0])
        self.process_data()
        
    def parMaxChanged(self,key):
        """
        Process a change of a minimum parameter value
        """
        p=self._parameterDict[key]
        p.setMaxValue(self.maxLEDict[key].text().toFloat()[0])
        self.process_data()
        
    def xAxisChange(self,key):
        """
        The user has selected a different x-axis
        """
        #Active slider of current x-axis
        self.slideDict[self._xName].setEnabled(True)
        self._xName=str(key)
        self.slideDict[self._xName].setEnabled(False)
        self.plot.set_axis_title(self.plot.X_BOTTOM,self._xName)
        self.process_data()
        
    def extend_widget(self):
        func_string=str(self.eqLE.text())
        self.getParametersAndFunction(func_string)
        self.eqLE.setEnabled(False)
        self.processButton.setEnabled(False)
        self.title="f=%s"%(func_string)
                
        #Check if _x in parameter list
        if("_x" in self._parameterDict.keys()):
            self.defineX("_x")  #Make _x the x-axis parameter
        else:
            self.defineX(self._parameterDict.keys()[0]) #Make the first parameter the x-axis
        #Loop over function parameters and setup sliders
        cnt=0
        glayout=QGridLayout()
        self.slideDict={} #to store parameter value sliders
        self.minLEDict={} #to store minimum parameter value LineEdits
        self.maxLEDict={} #to store maximum parameter value LineEdits
        self.valDict={} #to store labels containing parameter values
        self.xCBDict={} #To store x-axis selection checkbox
        palette = QPalette()
        palette.setColor(QPalette.Foreground,Qt.blue)
        key_lst=self._parameterDict.keys()
        key_lst.sort()
        self.xCombo=QComboBox()
        sizePol=QSizePolicy(QSizePolicy.Preferred,QSizePolicy.Preferred)
        for key in key_lst:
            self.xCombo.addItem(key)
            p=self._parameterDict[key]
            nameLabel=QLabel(p.name()+': ')
            glayout.addWidget(nameLabel,cnt,0)
            if(p.Scientific()):
                valLabel=QLabel("%5.4e"%(p.v()))
                minLE=QLineEdit("%5.4e"%(p.minValue()))
                #~ minLE.sizeHint(10)
                maxLE=QLineEdit("%5.4e"%(p.maxValue()))
            else:
                valLabel=QLabel("%6.3f"%(p.v()))
                minLE=QLineEdit("%6.3f"%(p.minValue()))
                #~ minLE.sizeHint(10)
                maxLE=QLineEdit("%6.3f"%(p.maxValue()))
            self.connect(minLE,SIGNAL('editingFinished ()'),\
                partial(self.parMinChanged,key))
            self.connect(maxLE,SIGNAL('editingFinished ()'),\
                partial(self.parMaxChanged,key))
            minLE.setSizePolicy(sizePol)
            maxLE.setSizePolicy(sizePol)
            self.minLEDict[key]=minLE
            self.maxLEDict[key]=maxLE
            valLabel.setPalette(palette)
            self.valDict[key]=valLabel    
            glayout.addWidget(valLabel,cnt,1)
            glayout.addWidget(minLE,cnt,2)
            glayout.addWidget(maxLE,cnt,4)
            sld = QSlider(Qt.Horizontal)
            sld.setRange(0,1000)
            
            s_value=int(1000*(p.v()-p.minValue())/(p.maxValue()-p.minValue()))
            sld.setValue(s_value)
            if(p.name()==self._xName):
                sld.setEnabled(False)
            self.connect(sld, SIGNAL('valueChanged(int)'),\
                partial(self.slideChange,key))
            self.slideDict[key]=sld
            #Connect this slider to the parameter
            glayout.addWidget(sld,cnt,3)
            cnt+=1
            
        self.vlayout.addLayout(glayout)
        hl=QHBoxLayout()
        self.xCombo.setCurrentIndex(self.xCombo.findText(self._xName))
        self.autoScaleCheckBox=QCheckBox("Auto scale y-axis")
        self.autoScaleCheckBox.setCheckState(self.autoScale)
        self.autoScaleCheckBox.setTristate(False)
        self.connect(self.autoScaleCheckBox,SIGNAL('stateChanged(int)'),\
            self.toggleAutoScale)
        hl.addWidget(self.autoScaleCheckBox)
        hl2=QHBoxLayout()
        xlab=QLabel("X-axis: ")
        hl2.addStretch()
        hl2.addWidget(xlab)
        hl2.addWidget(self.xCombo)
        hl.addLayout(hl2)
        self.connect(self.xCombo,SIGNAL('currentIndexChanged(QString)'),\
            self.xAxisChange)
        self.vlayout.addLayout(hl)
        self.setLayout(self.vlayout)
        
        self.plot.set_axis_title(self.plot.Y_LEFT,self.title)
        self.plot.set_axis_title(self.plot.X_BOTTOM,self._xName)
        
        self.process_data()
        
    def setup_widget(self, title):
        #---Create the plot widget:
        self.curvewidget = CurveWidget(self)
        self.curvewidget.register_all_curve_tools()
        self.curve_item = make.curve([], [], color='b')
        self.curvewidget.plot.add_item(self.curve_item)
        self.curvewidget.plot.set_antialiasing(True)
        self.plot = self.curvewidget.get_plot()
        
        font = QFont()
        font.setPointSize( 16 )
        
        self.plot.set_axis_font("left", font)
        self.plot.set_axis_font("bottom", font)
        #---
        self.eqLE=QLineEdit()
        self.eqLE.setPlaceholderText(\
            "e.g. sin(_x**2/_a+_y**2/_b), just start parameters with one underscore!")
        self.eqLE.selectAll()
        self.processButton=QPushButton("Process")
        self.connect(self.processButton,SIGNAL('clicked()'),\
            self.extend_widget)
        hlayout=QHBoxLayout()
        hlayout.addWidget(self.eqLE)
        hlayout.addWidget(self.processButton)
        self.vlayout = QVBoxLayout()
        self.vlayout.addWidget(self.curvewidget)
        self.vlayout.addLayout(hlayout)
        self.setLayout(self.vlayout)
        
    def process_data(self):
        self.y = self.calc()
        if(self.autoScale):
            self.plot.set_axis_limits(self.plot.Y_LEFT,min(self.y),max(self.y))
        self.update_curve()
        
    def update_curve(self):
        #---Update curve
        self.curve_item.set_data(self.xvec, self.y)
        self.curve_item.plot().replot()
        

class TestWindow(QWidget):
    def __init__(self):
        QWidget.__init__(self)
        self.setWindowTitle("FunctionPlotter(guiqwt)")
        self.setWindowIcon(get_icon('guiqwt.svg'))
        hlayout = QHBoxLayout()
        self.setLayout(hlayout)
        
    def add_plot(self, title):
        self.widget = TFunctionUI(self)
        self.widget.setup_widget(title)
        self.layout().addWidget(self.widget)

if __name__ == "__main__":
    app = QApplication([])
    win = TestWindow()
    win.add_plot("")
    win.show()
    app.exec_()

Locating the site-packages folder from within python

Nice way to find where the site-packages folder is located for your linux distribution from within python itself (found here)

import site
print site.getsitepackages()

Apparently not available in older python version.