nmrglue.pipe

Functions for reading and writing NMRPipe binary files and table (.tab) files

NMRPipe file structure is described in the NMRPipe man pages and fdatap.h

This modules is imported as nmrglue.pipe and can be called as such.

High-Level Functions

These are the functions most users will use from the pipe module.

nmrglue.fileio.pipe.read(filename)

Read a NMRPipe binary file returning a dic,data pair.

For standard multi-file 3D/4D NMRPipe data sets, filename should be a filemask (for example “/ft/test%03d.ft3”) with a “%” formatter. If only one file of a 3D/4D data set is provided only that 2D slice of the data is read (for example “/ft/test001.ft3” results in a 2D data set being read).

NMRPipe data streams stored as files (one file 3D/4D data sets made using xyz2pipe) can be read by providing the file name of the stream. The entire data set is read into memory.

nmrglue.fileio.pipe.read_lowmem(filename)

Read a NMRPipe binary file with minimal memory usage.

For standard multi-file 3D/4D NMRPipe data sets, filename should be a filemask (for example “/ft/test%03d.ft3”) with a “%” formatter. If only one file of a 3D/4D data set is provided only that 2D slice of the data is read (for example “/ft/test001.ft3” results in a 2D data set being read).

NMRPipe data streams stored as files (one file 3D/4D data sets made using xyz2pipe) can be read by providing the file name of the stream.

nmrglue.fileio.pipe.write(filename, dic, data, overwrite=False)

Write a NMRPipe file

For 3D data if filename has no ‘%’ formatter then the data is written as a 3D NMRPipe data stream. When the ‘%’ formatter is provided the data is written out as a standard NMRPipe 3D multi-file 3D.

For 4D data, filename can have one, two or no ‘%’ formatters resulting in a single index file (test%03d.ft), two index file(test%02d%03d.ft), or one file data stream (test.ft4).

dic[“FDPIPEFLAG”] is not changed or checked when writing, please check that this value is 0.0 for standard non-data stream files, and 1.0 for data stream files or an file may be written with an incorrect header.

Set overwrite to True to overwrite files that exist.

nmrglue.fileio.pipe.write_lowmem(filename, dic, data, overwrite=False)

Write a NMRPipe file using minimal memory

nmrglue.fileio.pipe.read_table(filename)

Read a NMRPipe database table (.tab) file.

Parameters:

• filename Name of file to read

Returns: pcomments,pformat,rec

• pcomments List of NMRPipe comment lines.
• pformats List of NMRPipe table column formats strings.
• rec Records array with named fields.

nmrglue.fileio.pipe.write_table(filename, pcomments, pformats, rec, overwrite=False)

Write a NMRPipe database table (.tab) file.

Parameters:

• filename Name of file to write to.
• pcomments List of NMRPipe comment lines.
• pformats List of NMRPipe table column formats strings.
• rec Records array of table.
• overwrite Set True to overwrite file if it exists.

nmrglue.fileio.pipe.make_uc(dic, data, dim=-1)

Make a unit conversion object

nmrglue.fileio.pipe.guess_udic(dic, data)

Guess parameter of universal dictionary from dic,data pair

nmrglue.fileio.pipe.create_dic(udic, datetimeobj=datetime.datetime(2011, 4, 6, 16, 39, 24, 713603))

Crate a NMRPipe dictiony from universal dictionary

Parameters:

• udic Universal dictionary
• datetimeobj datetime object
• user Name of user

Does not update dictionary keys that are unknown such as MIN/MAX, apodization and processing parameters, sizes in none-current domain. Also rounding of parameter is different than NMRPipe.

nmrglue.fileio.pipe.datetime2dic(dt, dic)

Add datatime object to dictionary

nmrglue.fileio.pipe.dic2datetime(dic)

Create a datetime object from dictionary

High-Level Classes

These are the classes most users will use from the pipe module.

class nmrglue.fileio.pipe.iter3D(filemask, in_lead='x', out_lead='DEFAULT')

Object which allows for graceful iteration over 3D NMRPipe files

iter3D.iter() returns a (dic,plane) tuple which can be written using the x.writeplane function.

When processing 3D files with iter3D object(s) the following dictionary parameters may not have the same values as NMRPipe processing scripts return:

FDSLICECOUNT and

FDMAX,FDDISMAX,FDMIN,FDDISPMIN when FDSCALEFLAG == 0

Example:

#3D data processing
xiter = iter3D("data/test%03d.fid","x","x")
for dic,YXplane in xiter:
    # process X and Y axis
    xiter.write("ft/test%03d.ft2",YXplane,dic)
ziter = iter3D("ft/test%03d.ft2","z","z")
for dic,XZplane in ziter:
    # process Z axis
    ziter.write("ft/test%03d.ft3",XZplane,dic)

Low-Level Functions

These functions are typically not used directly by users. They are called by high level functions. Developers and user who want fine control over pipe files will be interested in these functions.

nmrglue.fileio.pipe.create_data(data)

Create a NMRPipe data array (recast into float32 or complex64)

nmrglue.fileio.pipe.add_axis_to_dic(dic, adic, n)

Add an axis to NMRPipe dictionary

n is 0,1,2,... (0 is direct dim, 1 first indirect...)

nmrglue.fileio.pipe.create_empty_dic()

Creates a nmrpipe dictionary with default values

nmrglue.fileio.pipe.read_1D(filename)

Read a 1D NMRPipe binary file returning a dic,data pair

nmrglue.fileio.pipe.read_2D(filename)

Read a 2D NMRPipe file or NMRPipe data stream returning a dic,data pair

nmrglue.fileio.pipe.read_lowmem_2D(filename)

Read a 2D NMRPipe file or NMRPipe data stream with minimal memory usage

nmrglue.fileio.pipe.read_stream(filename)

Read a NMRPipe data stream (one file 3D or 4D files)

nmrglue.fileio.pipe.read_lowmem_stream(filename)

Read a NMRPipe data stream with minimal memory usage

nmrglue.fileio.pipe.read_3D(filemask)

Read a 3D NMRPipe binary file returning a dic,data pair.

nmrglue.fileio.pipe.read_lowmem_3D(filemask)

Read a 3D NMRPipe binary file with minimal memory usage.

nmrglue.fileio.pipe.read_4D(filemask)

Read a 3D NMRPipe binary file returning a dic,data pair.

This function should not be used to read NMRPipe data streams stored in a single file (one file 3D/4D data sets made using xyz2pipe), read_2D should be used.

nmrglue.fileio.pipe.read_lowmem_4D(filemask)

Read a NMRPipe binary file with minimal memory usage.

This function should not be used to read NMRPipe data streams stored in a single file (one file 3D/4D data sets made using xyz2pipe), read_lowmem_2D should be used.

nmrglue.fileio.pipe.write_single(filename, dic, data, overwrite=False)

Write data to a single NMRPipe file from memory. This write 1D and 2D files completely as well as NMRPipe data streams. 2D planes of 3D and 4D files should be written with this function.

nmrglue.fileio.pipe.write_3D(filemask, dic, data, overwrite=False)

Write a standard multi-file 3D NMRPipe file

nmrglue.fileio.pipe.write_4D(filemask, dic, data, overwrite=False)

Write a one or two index 4D NMRPipe file

nmrglue.fileio.pipe.write_lowmem_2D(filename, dic, data, overwrite=False)

Write a 2D NMRPipe file using minimal memory (trace by trace)

nmrglue.fileio.pipe.write_lowmem_3D(filename, dic, data, overwrite=False)

Write a standard multi-file 3D NMRPipe file using minimal memory (trace by trace)

MIN/MAX parameters are not updated in the NMRPipe headers.

nmrglue.fileio.pipe.write_lowmem_3Ds(filename, dic, data, overwrite=False)

Write 3D NMRPipe data stream file using minimal memory (trace by trace)

nmrglue.fileio.pipe.write_lowmem_4D(filename, dic, data, overwrite=False)

Write a standard multi-file (single or double index 4D NMRPipe file using minimal memory (trace by trace).

MIN/MAX parameters are not updated in the NMRPipe headers.

nmrglue.fileio.pipe.write_lowmem_4Ds(filename, dic, data, overwrite=False)

Write 4D NMRPipe data stream file using minimal memory (trace by trace)

nmrglue.fileio.pipe.put_fdata(fh, fdata)

Put fdata to NMRPipe file described by file object fh

nmrglue.fileio.pipe.put_trace(fh, trace)

Put 1D trace (real or complex) to NMRPipe file described by file object fh

nmrglue.fileio.pipe.put_data(filename, fdata, data, overwrite=False)

Put fdata and data to 2D NMRPipe.

nmrglue.fileio.pipe.write_slice_3D(filemask, dic, data, shape, (sz, sy, sx))

Write a slice of a 3D data array to file

Opens (or if necessary creates) 2D NMRPipe file(s) to write data, where total 3D file size is given by shape.

Parameters: * filemask String with single formatting operator (%) * data 3D array of data * dic Dictionary to write when/if files are created * shape 3-tuple of integers indicating the overall matrix shape * (sz,sy,sx) 3-tuple of slice object which specify location of data

This function memmaps 2D NMRPipe files for speed. It only writes dictionaries to file when created, leaving them unmodified if the file exists.

Only error checking is that data is 3D.

Users are not expected to use this function, rather use the iter3D object

nmrglue.fileio.pipe.pack_complex(data)

Pack inteleaved real,imag array into complex array

nmrglue.fileio.pipe.transpose_3D(dic, data, (a1, a2, a3)=(2, 1, 0))

Transpose pipe_3d object and dictionary

nmrglue.fileio.pipe.find_shape(dic)

Find the shape (tuple) of data in a NMRPipe file from dictionary

1-tuple is returned for 1D data, 2-tuple for 2D and non-stream 3D/4D data, 3-tuple or 4-tuple for stream 3D/4D data.

The last dimension of the tuple is length of the data in the file, the actual length of the data matrix may be half of this if the data is complex.

nmrglue.fileio.pipe.reshape_data(data, shape)

Reshape data or return 1D data after warning

nmrglue.fileio.pipe.unshape_data(data)

Returns 1D version of data

nmrglue.fileio.pipe.unappend_data(data)

Returns complex data with last axis (-1) unappended

Data should have imaginary data vector appended to real data vector

nmrglue.fileio.pipe.append_data(data)

Return data with last axis (-1) appeneded

Data should be complex

nmrglue.fileio.pipe.fdata2dic(fdata)

Convert a fdata array to fdata dictionary

Converts the raw 512x4-byte NMRPipe header into a python dictionary with keys as given in fdatap.h

nmrglue.fileio.pipe.dic2fdata(dic)

Converts a NMRPipe dictionary into an array

nmrglue.fileio.pipe.get_fdata(filename)

Get an array of length 512 holding NMRPipe header

nmrglue.fileio.pipe.get_data(filename)

Get array of data

nmrglue.fileio.pipe.get_fdata_data(filename)

Get fdata and data array, returns (fdata,data)

nmrglue.fileio.pipe.get_trace(fhandle, ntrace, pts, bswap, cplex)

Get a single trace from a NMRPipe file

Parameters:

• fhandle File object of open NMRPipe file.
• ntrace Trace numbers (starting from 0).
• pts Number of points in trace, R|I.
• bswap True/False to perform byteswap on trace.
• cplex True/False to unappend imaginary data.

Low-Level Classes

class nmrglue.fileio.pipe.pipe_2d(filename, order=(0, 1))

Emulate a numpy.ndarray objects without loading data into memory for low memory reading of 2D NMRPipe files.

• slicing operations return ndarray objects.
• can iterate over with expected results.
• transpose and swapaxes methods create a new objects with correct axes ordering.
• has ndim, shape, and dtype attributes.

class nmrglue.fileio.pipe.pipe_3d(filemask, order=(0, 1, 2), fcheck=False)

Emulate a numpy.ndarray objects without loading data into memory for low memory reading of 3D NMRPipe files (multiple file data sets)

• slicing operations return ndarray objects.
• can iterate over with expected results.
• transpose and swapaxes methods create a new objects with correct axes ordering.
• has ndim, shape, and dtype attributes.

class nmrglue.fileio.pipe.pipestream_3d(filename, order=(0, 1, 2))

Emulate a numpy.ndarray objects without loading data into memory for low memory reading of 3D NMRPipe data stream (one file 3D):

• slicing operations return ndarray objects.
• can iterate over with expected results.
• transpose and swapaxes methods create a new objects with correct axes ordering.
• has ndim, shape, and dtype attributes.

class nmrglue.fileio.pipe.pipe_4d(filemask, order=(0, 1, 2, 3), fcheck=False)

Emulate a numpy.ndarray objects without loading data into memory for low memory reading of single/two index 4D NMRPipe files.

• slicing operations return ndarray objects.
• can iterate over with expected results.
• transpose and swapaxes methods create a new objects with correct axes ordering.
• has ndim, shape, and dtype attributes.

class nmrglue.fileio.pipe.pipestream_4d(filename, order=(0, 1, 2, 3))

Emulate a numpy.ndarray objects without loading data into memory for low memory reading of 4D NMRPipe data stream (one file 4D):

• slicing operations return ndarray objects.
• can iterate over with expected results.
• transpose and swapaxes methods create a new objects with correct axes ordering.
• has ndim, shape, and dtype attributes.