Plugins¶

Any new hardware has to be included in PyMoDAQ as a python plugin. This is a script containing a python object following a particular template and behaviour and inheriting from a base class. Plugins are articulated given their type: Moves or Viewers and for the last their main dimensionality: 0D, 1D or 2D. It is recommended to start from the template plugins (daq_move_Template, daq_NDviewer_Template, see below) and then check from other examples (pymodaq_plugins repository) the proper way of writing a plugin. You will find below some information on the how to but comparison with existing ones will be beneficial.

Installation¶

All plugins are located in the pymodaq_plugins repository on github. This constitutes a python package that will be installed together with PyMoDAQ (or manually install it if you don’t want the up to date version). PyMoDAQ will therefore call the plugins from the pymodaq_plugins package installed on the site_packages location in python distribution.

List of plugins¶

Table 1 Plugins list¶
Name :	Type :	Manufacturer :	Tested Models :	Rqs:
Mock	DAQ_Move	Simulation purpose
Conex	DAQ_Move	Newport	CONEX-AGAP controller	using the .Net ConexAGAPCmdLib library
Holoeye	DAQ_Move	Holoeye	LC2012	using slmdisplaysdk python package
Kinesis	DAQ_Move	Thorlabs	K10RC1 Stepper Motor Rotation Mount	using the .Net Kinesis software
Kinesis_Flipper	DAQ_Move	Thorlabs	MFF101 Motorized Filter Flip Mounts	using the .Net Kinesis software
PI	DAQ_Move	Physik Instrumente	E816, Mercury stepper, C-863	using the pipython package
PI_MMC	DAQ_Move	Physik Instrumente	C-862	using the MMC.dll library
PiezoConcept	DAQ_Move	PiezoConcept	Bio200	pyvisa, piezoconcept wrapper
Spectro270M	DAQ_Move	JobinYvon		not working yet
OrsaySTEM	DAQ_Move	Orsay LPS lab	OUDS2	used to do Scanning Transmission Electron Microscopy using the custom OUDS electronics. Move the electronic beam
Mock	DAQ_0DViewer	Simulation purpose
NIDAQmx	DAQ_0DViewer	National Instruments	various not too old data acquisition cards	using the daqmx package (need the daqmx dll), counting and Analog in average to produce 0D data
Keithley_Pico	DAQ_0DViewer	Keithley	648X, 6430, 6514	using pyvisa protocol
Lockin7270	DAQ_0DViewer	Ametek Signal Recovery	7270 DSP	using pyvisa protocol
LockinSR830	DAQ_0DViewer	Stanford Reasearch Systems	SR830 DSP Lockin amplifier	using pyvisa protocol
Mock	DAQ_1DViewer	Simulation purpose
NIDAQmx	DAQ_1DViewer	National Instruments	various not too old data acquisition cards	using the daqmx package (need the daqmx dll), Analog in
OceanOptics	DAQ_1DViewer	OceanOptics	USB2000, USB2000+, USB4000…	using the .Net NETOmniDriver-NET40
Picoscope	DAQ_1DViewer	PicoTechnology	picoscope 5000A	custom wrapper from the dll (not nicely written but working)
Shamrock	DAQ_1DViewer	Andor	all shamrock spectrometers	using a custom wrapper of the dll
Tektronix	DAQ_1DViewer	Tektronix	MDO30xx	using pyvisa, light plugin to only get channels from screen
TH260	DAQ_1DViewer	Picoquant	Timeharp260P	using a custom wrapper of the dll
Mock	DAQ_2DViewer	Simulation purpose
AndorCCD	DAQ_2DViewer	Andor	CCD Idus (should work with all CCDs)	custom wrapper of the sdk2 dll
OpenCVCam	DAQ_2DViewer	webcams		using windows universal driver and builtin functionalities from opencv-python
OrsayCamera	DAQ_2DViewer	Princeton/Ropers	PIXIS256	custom wrapper from Orsay LPS lab to interface cameras using 64 bit PICAM library
OrsayStem	DAQ_2DViewer			used to do Scanning Transmission Electron Microscopy using the custom OUDS electronics. acquire STEM images and SPIM data cubes
TCP_GRABBER	DAQ_2DViewer	/	/	TCP/IP server protocol to communicate with non local hardware.
FLIM	DAQ_2DViewer	Picoquant/piezoconcept	Timeharp260P + Bio200	Plugins for Fluorescence lifetime imaging microscopy: cartography (2D using the Bio200) of fluorescence lifetime
GenICam	DAQ_2DViewer	Generic protocol using GenTL producers	Sentech STC-MBS500U	Using the GenICam protocol many complient industrial cameras can be used without any use of their own libraries. Matrix Vision GenTL producer should be installed (see plugin doc)
TIS	DAQ_2DViewer	The Imaging Source	various USB models	The plugin uses the tisgrabber.dll with the pyicic python wrapper package

See up-to-date database: https://docs.google.com/spreadsheets/d/1wfMfvLwTitZd2R2m1O5i6wVEaX1lJBahP2HUbxVdidg

Naming convention:¶

For the plugin to be properly recognised by PyMoDAQ, its location and name must follow these rules:

An actuator plugin (name: xxxx) will be a script whose name is daq_move_Xxxx (notice first X letter is capital)
The plugin class within the script will be named DAQ_Move_Xxxx (notice the capital letters here as well)
the script will be located within pymodaq_plugins installed package tree in C:\WPy-3710\...\pymodaq_plugins\daq_move_plugins\
A detector plugin of dimensionality N (N=0, 1 or 2) (name: xxxx) will be a script whose name is daq_NDviewer_Xxxx (notice first X letter is capital, and replace N by 0, 1 or 2)
The plugin class within the script will be named DAQ_NDViewer_Xxxx (notice the capital letters here as well)
the script will be located within pymodaq_plugins installed package tree in C:\WPy-3710\...\pymodaq_plugins\daq_viewer_plugins\plugins_ND (replace N by 0, 1 or 2)

Hardware Settings¶

An important feature similar for all modules is the layout as a tree structure of all the hardware parameters. These settings will appear on the UI as a tree of parameters with a title and different types, see Fig. 26. On the module side, they will be instantiated as a list of dictionaries and later exist in the object self.settings. This object inherits from on the Parameter object defined in pyqtgraph.

Fig. 26 Typical hardware settings represented as a tree structure (here from the daq_2Dviewer_AndorCCD plugin)

Here is an example of such a list of dictionaries corresponding to Fig. 26:

[{'title': 'Dll library:', 'name': 'andor_lib', 'type': 'browsepath', 'value': libpath},
 {'title': 'Camera Settings:', 'name': 'camera_settings', 'type': 'group', 'expanded': True, 'children': [
     {'title': 'Camera SN:', 'name': 'camera_serialnumber', 'type': 'int', 'value': 0, 'readonly': True},
     {'title': 'Camera Model:', 'name': 'camera_model', 'type': 'str', 'value': '', 'readonly': True},
     {'title': 'Readout Modes:', 'name': 'readout', 'type': 'list', 'values': ['FullVertBinning','Imaging'], 'value': 'FullVertBinning'},
     {'title': 'Readout Settings:', 'name': 'readout_settings', 'type': 'group', 'children':[
         {'title': 'single Track Settings:', 'name': 'st_settings', 'type': 'group', 'visible': False, 'children':[
             {'title': 'Center pixel:', 'name': 'st_center', 'type': 'int', 'value': 1 , 'default':1, 'min':1},
             {'title': 'Height:', 'name': 'st_height', 'type': 'int', 'value': 1 , 'default':1, 'min':1},
             ]},]}]}]

The list of available types of parameters :module:`custom_parameter_tree` (defined in pymodaq.daq_utils.custom_parameter_tree.py) is:

group : “camera settings” on Fig. 26 is of type group
int : settable integer (SpinBox_Custom object)
float : settable float (SpinBox_Custom object)
str : a QLineEdit object (see Qt5 documentation)
list : “Readout Modes” Fig. 26 is a combo box
bool : checkable boolean
led : checkable boolean in the form of a green (True) of red (False) led
date_time : a QDateTime object (see Qt5 documentation)
date : a QDate object (see Qt5 documentation)
time : a QTime object (see Qt5 documentation)
slide : a combination of a slide and spinbox for floating point values
itemselect : an object to easily select one or more items among a few
browsepath: a text area and a pushbutton to select a given path or file
text : a text area (for comments for instance)

Important: the name key in the dictionnaries must not contain any space, please use underscore if necessary!

Once the module is initialized, any modification on the UI hardware settings will be send to the plugin through the commit_settings method of the plugin class and illustrated below (still from the daq_2Dviewer_AndorCCD plugin). The param method argument is of the type Parameter (from pyqtgraph):

def commit_settings(self,param):
    """
        | Activate parameters changes on the hardware from parameter's name.
    """
    try:
        if param.name()=='set_point':
            self.controller.SetTemperature(param.value())

        elif param.name() == 'readout' or param.name() in custom_parameter_tree.iter_children(self.settings.child('camera_settings', 'readout_settings')):
            self.update_read_mode()

        elif param.name()=='exposure':
            self.controller.SetExposureTime(self.settings.child('camera_settings','exposure').value()/1000) #temp should be in s
            (err, timings) = self.controller.GetAcquisitionTimings()
            self.settings.child('camera_settings','exposure').setValue(timings['exposure']*1000)
        elif param.name() == 'grating':
            index_grating = self.grating_list.index(param.value())
            self.get_set_grating(index_grating)
            self.emit_status(ThreadCommand('show_splash', ["Setting wavelength"]))
            err = self.controller.SetWavelengthSR(0, self.settings.child('spectro_settings','spectro_wl').value())
            self.emit_status(ThreadCommand('close_splash'))

DAQ Move plugin template¶

An actuator plugin is a python class inheriting from a base class. Let’s say you want to create the template plugin. You will first create a daq_move_Template.py file in the \pymodaq_plugins\daq_move_plugins folder. The plugin class will be called DAQ_Move_Template.

See daq_move_Template.py for a detailed template.

DAQ Viewer plugin template¶

A detector plugin is a python class inheriting from a base class. Let’s say you want to create the template plugin. You will first create a daq_NDviewer_Template.py file in the \pymodaq_plugins\daq_viewer_plugins\plugins_ND\ folder (with N=0, 1 or 2 depending the data dimensionality of your detector). The plugin class will be called DAQ_NDViewer_Template.

See daq_NDviewer_Template.py for a detailed template.

Emission of data¶

When data are ready (see Data ready? to know about that), the plugin has to notify the viewer module in order to display data and eventually save them. For this PyMoDAQ use two types of signals (see pyqtsignal documentation for details):

data_grabed_signal_temp
data_grabed_signal

They both emit the same type of signal but will trigger different behaviour from the viewer module. The first is to be used to send temporary data to update the plotting but without triggering anything else (so that the DAQ_Scan still awaits for data completion before moving on). It is also used in the initialisation of the plugin in order to preset the type and number of data viewers displayed by the viewer module. The second signal is to be used once data are fully ready to be send back to the user interface and further processed by DAQ_Scan or DAQ_Viewer instances. The code below is an example of emission of data:

x_axis = dict(label='Wavelength', units= "nm", data = vector_X)
self.data_grabed_signal.emit([OrderedDict(name='Camera',data=[data2D_0, data2D_1,...], type='Data2D',x_axis=vector_X,y_axis=vector_Y),
                              OrderedDict(name='Spectrum',data=[data1D_0, data1D_1,...], type='Data1D',x_axis=vector_X),
                              OrderedDict(name='Current',data=[data0D_0, data0D_1,...], type='Data0D'),
                              OrderedDict(name='Datacube',data=[dataND_0, dataND_1,...], type='DataND', nav_axes=[0,2]),
                                        ])

Such an emitted signal would trigger the initialization of 4 data viewers in the viewer module. One for each OrderedDict in the emitted list. The type of data viewer will be determined by the type key value while its name will be set to the name key value. The data key value is also a list of numpy arrays, their shape should be adequate with the type key of the dictionary. Each array will generate one channel within the corresponding viewer. Here is the detailed list of the possible keys:

name: will display the corresponding value on the viewer dock
type: will set the viewer type (0D, 1D, 2D or multi-dimensional ND). The ND viewer will be able to deal with data dimensionality up to 4)
data: list of numpy array. Each array shape should correspond to the type
x_axis: either a numpy 1D array representing the x axis of the detector (wavelength for a spectrometer for instance, default is pixel number) or a dictionnary containing various fields to set the axis labels, units on the viewer (see code above)
y_axis: either a numpy 1D array representing the y axis of the detector (only for 2D detector) (default is pixel number) or a dictionnary containing various fields to set the axis labels, units on the viewer (see code above)
nav_axis: in case of a ND data viewer, will be the index of the navigation axis, see ND Viewer

Data ready?¶

One difficulty with these viewer plugins is to determine when data is ready to be read from the controller and then to be send to the user interface for plotting and saving. There are a few solutions:

synchronous: The simplest one. When the grab command has been send to the controller (let’s say to its grab_sync method), the grab_sync method will hold and freeze the plugin until the data are ready. The Mock or Tektronix modules work like this.
asynchronous: There are 2 ways of doing asynchronous waiting. The first is to poll the controller state to check if data are ready within a loop. This polling could be done with a while loop but if nothing more is done then the plugin will still be freezed, except if one process periodically the Qt queue event using QtWidgets.QApplication.processEvents() method. The polling can also be done with a timer event, firing periodically a check of the data state (ready or not). Finally, the nicest/hardest solution is to use callbacks (if the controller provides one) and link it to a emit_data method.

Synchronous example:¶

The code below illustrates the poll method using a loop:

def poll_data(self):
    """
    Poll the current data state
    """
    sleep_ms=50
    ind=0
    data_ready = False
    while not self.controller.is_ready():
        QThread.msleep(sleep_ms)

        ind+=1

        if ind*sleep_ms>=self.settings.child(('timeout')).value():

            self.emit_status(ThreadCommand('raise_timeout'))
            break

        QtWidgets.QApplication.processEvents()
    self.emit_data()

Asynchronous example:¶

The code below is derived from daq_Andor_SDK2 (in andor hardware folder) and shows how to create a thread waiting for data ready and triggering the emission of data

class DAQ_AndorSDK2(DAQ_Viewer_base):

    callback_signal = QtCore.pyqtSignal() #used to talk with the callback object
    ...

    def ini_camera(self):
        ...
        callback = AndorCallback(self.controller.WaitForAcquisition) # the callback is linked to the controller WaitForAcquisition method
        self.callback_thread = QtCore.QThread() #creation of a Qt5 thread
        callback.moveToThread(self.callback_thread) #callback object will live within this thread
        callback.data_sig.connect(self.emit_data)  # when the wait for acquisition returns (with data taken), emit_data will be fired

        self.callback_signal.connect(callback.wait_for_acquisition) #
        self.callback_thread.callback = callback
        self.callback_thread.start()

    def grab(self,Naverage=1,**kwargs):
        ...
        self.callback_signal.emit()  #trigger the wait_for_acquisition method

def emit_data(self):
    """
        Function used to emit data obtained by callback.
    """
    ...
    self.data_grabed_signal.emit([OrderedDict(name='Camera',data=[np.squeeze(self.data.reshape((sizey, sizex)).astype(np.float))], type=self.data_shape)])


class AndorCallback(QtCore.QObject):

    data_sig=QtCore.pyqtSignal()
    def __init__(self,wait_fn):
        super(AndorCallback, self).__init__()
        self.wait_fn = wait_fn

    def wait_for_acquisition(self):
        err = self.wait_fn()

        if err != 'DRV_NO_NEW_DATA': #will be returned if the main thread called CancelWait
            self.data_sig.emit()

Documentation from Andor SDK concerning the WaitForAcquisition method of the dll:

unsigned int WINAPI WaitForAcquisition(void)

WaitForAcquisition can be called after an acquisition is started using StartAcquisition to put the calling thread to sleep until an Acquisition Event occurs.

It will use less processor resources than continuously polling with the GetStatus function. If you wish to restart the calling thread without waiting for an Acquisition event, call the function CancelWait.

Hardware averaging:¶

By default, if averaging of data is needed the Viewer module will take care of it software wise. However, if the hardware controller provides an efficient method to do it (that will save time) then you should set the class field hardware_averaging to True.

class DAQ_NDViewer_Template(DAQ_Viewer_base):
"""
 Template to be used in order to write your own viewer modules
"""
    hardware_averaging = True #will use the accumulate acquisition mode if averaging
    #is True else averaging is done software wise

Hardware needed files¶

If you are using/referring to custom python wrappers/dlls… within your plugin and need a place where to copy them in PyMoDAQ, then use the \pymodaq_plugins\hardware folder. For instance, the daq_2Dviewer_AndorCCD plugin need various files stored in the andor folder (on github repository). I would therefore copy it as \pymodaq_plugins\hardware\andor and call whatever module I need within (meaning there is a __init__.py file in the andor folder) as:

#import controller wrapper
from pymodaq_plugins.hardware.andor import daq_AndorSDK2 #this import the module DAQ_AndorSDK2 containing classes, methods...
#and then use it as you see fit in your module

Specific TCP/IP plugin¶

It is possible to use a TCP/IP plugin in order to communicate with a distant client. For instance, the module daq_2Dviewer_TCP_GRABBER is one such example. It inherits from the base DAQ_TCP_Server class:

from pymodaq.daq_viewer.utility_classes import DAQ_TCP_server

This plugin is still experimental and focused on one particular relation with a client. Please open an issue on github if you have specific need and/or propositions.

How to contribute?¶

If you wish to develop a plugin specific to a new hardware not present on the github repo (and I strongly encourage you to do so!!), you will have to follow the rules as stated above. However, the best practice would be to fork pymodaq_plugins repository. On windows, you can use Github Desktop. Then you can manually install the forked package (typically using pip install -e . from winpython command line where you cd within the forked package. This command will kind of install the package but any change you apply on the local folderwill be applied on the package. Once you’re ready with a working plugin, you can then push your branch that will be merged with the main branch after validation.