Medical_Imaging_Software_Development_-_Statement_of_Work.pdf

STATEMENT OF WORK

“Medical Imaging Software Development”

1. INTRODUCTION / OVERVIEW:

1.1. Background:

The National Heart, Lung, and Blood Institute (NHLBI) has a medical imaging research program, which encompasses acquisition, reconstruction, post-processing, and clinical application of X-ray, CT, and MRI. As part of this broad program, several open source software packages are being developed and maintained by NHLBI scientists. Specifically, the Laboratory of Imaging Technology (LIT), under the leadership of Dr. Michael S. Hansen, is the driving force behind the ISMRM Raw Data (ISMRMRD) format (https://ismrmrd.github.io), which is the community standard for representation and storage of MRI raw data. This data format is used in a variety of image reconstruction and processing tools. The LIT is also actively developing and distributing an open source image reconstruction framework known as the Gadgetron (https://gadgetron.github.io). This framework includes a streaming pipeline architecture for efficient data processing and a set of toolboxes that provide image reconstruction components such as filters, transformations, and optimizers. In addition to these major software projects, the LIT also maintains various tools for integration with proprietary medical imaging instruments.

As these tools grow in scope and popularity, there is a need for continuous restructuring and enhancement of the software packages. The introduction of enhancements must be done with a view to both current and future applications that need to be supported.

1.2. Scope:

The NHLBI seeks a contractor to carry out key enhancements of the ISMRMRD format that are needed to accommodate new data types and formalize and define certain work flows. A contractor is also needed to implement enhancements in the Gadgetron framework to modernize some older sections of code and to accommodate the changes implemented in the ISMRMRD format.

2. REQUIREMENTS:

2.1. Contractor Requirements:

The work is divided into two major tasks. The goal of Task 1 is to implement enhancements that are needed for the next version (described in this document as ISMRMRD 2.x) of the ISMRMRD format, and the goal of Task 2 is to enhance the Gadgetron to use the new features of the ISMRMRD format and to modernize some code segments that are mainly related to General Purpose Graphics Processing Unit (GPGPU) software.

Task 1. ISMRMRD 2.x

Task 1.1. New ISMRMRD Data Types The ISMRMRD currently supports storage of MRI Raw Data acquisitions (NMR spectrometer data stored readout by readout) and header information (in XML format). There is also a rudimentary image format used to store and transmit reconstructed images. While many MRI acquisitions can be transformed into images using the MRI acquisition alone, there is often need to incorporate other types of data. Physiological telemetry data (ECG, respiratory, etc.) may be needed to sort or correct data. Devices that monitor patient bulk motion, device positioning, or other experimental details may also provide input to the data processing pipeline and it may be necessary to provide calibration data (e.g., receiver coil sensitivity maps, gradient distortion coefficients, etc.), which would typically be available as multi-dimensional array data. Finally, modern MRI instruments may include multiple spectrometers (independent receiver systems), e.g., one for the imaging acquisition and another acquiring data from dynamic field monitoring probes. It is necessary to be able to represent all of these different types of data in the ISMRMRD format. The specific task items include:

1.1.1. Support for data source identification

The new version of the data format should have a mechanism for data source identification. As an example, if MRI raw data can be acquired with multiple spectrometers, there should be a way of indicating the source. This feature could be implemented as a data source field associated with every item that can be stored (or transmitted, see below).

1.1.2. Data types for generic waveform data.

A new data type capable of storing arbitrary waveform data should be implemented. These waveforms would be stored (and transmitted) in chunks and appropriate time stamps should be implemented that will allow for an application to reassemble a contiguous time varying waveform from the chunks of waveform data.

1.1.3. Data types for physiological monitoring data.

A special case of the generic waveform data is physiological monitoring (telemetry) data. These types of data include: a) multiple ECG leads, b) respiratory monitoring devices, c) pulse oximetry, and d) blood pressure transducer data. Multiple channels of data for each type should be supported. The implementation of these data types should include flags that will enable tagging of the physiological waveform data to indicate if this data is actual monitoring data (while acquisition is ongoing) or it is calibration data acquired prior to the acquisition.

1.1.4. Data types for sequence waveforms.

It is desired to be able to store the actual pulse sequence waveforms that were played out during the sequence. These waveforms are special cases of the generic waveform data. It is required to be able to store multiple channels of radio frequency (RF) excitation waveforms, multiple gradient waveforms, and waveforms that indicate when the sampling occurs (ADC waveforms).

1.1.5. Data types for calibration data – tagged multi-dimensional arrays A generic system for storing (and transmitting) multi-dimensional array data (possibly acquired and processed in previous acquisition). Examples of such data include: a) coil sensitivity maps, b) gradient distortion coefficients or maps, c) B0 maps, d) B1 maps, and e) gradient impulse response functions. These array structures need to be accompanied by appropriate meta data, which would include information about orientation, field of view, etc. when appropriate along with tags that would allow a reconstruction application to determine the purpose of the data (what type of data is it).

1.1.6. Clear definition of the ISMRMRD Image data format

The current implementation of ISMRMRD includes a definition of a rudimentary image format, which is currently used to store and transmit reconstruction results. This image format adds to a large number of existing open image data formats and it would be desirable if one of the current standards could be adopted as image representation. The contractor shall explore whether this format can be replaced with an existing open standard for storing images (e.g., Nifti, Analyze, etc.) and if so should implement any changes required for this transition. If the current approach of maintaining an ISMRMRD image format is the best approach, then the image format should be clearly defined including a list of commonly used meta data tags available for such images.

Task 1.2. File storage of ISMRMRD 2.x datasets The current version of ISMRMRD uses HDF5 as a file container for the existing data types. The next version should do the same. With the increase in the number of data types new requirements arise and changes to the file storage must be made to accommodate these changes. The implemented file storage scheme should achieve two goals; a) it should be possible to read specific types of data from the file without needing to traverse the entire dataset. As an example, one should be able to read only physiological waveforms without touching the MRI raw data itself. b) it should be possible to read the data objects (which may have different types) in the order in which they were added to the dataset.

1.2.1. Storage of data types in separate variables

Each type of data, e.g. MRI raw data acquisitions, ECG waveforms, etc., should be stored in its own variable. These variables should be separated for different data sources, e.g., MRI acquisitions from instrument 0 would be stored in ‘/dataset/data0’ while acquisitions from instrument 1 would be stored in ‘/dataset/data1’.

1.2.2. Index variable

There should be an index variable which will maintain the chronological order in which the data was acquired. If one wishes to read the data from the file in the order it was produced from the various instrument components, one would read the index variable in order and this variable would point to entries in the other variables in the dataset.

Task 1.3. ISMRMRD Support Libraries The current version of ISMRMRD has support libraries (APIs) in C++, Matlab, and Python. The purpose of these libraries is mainly to read and write the data types from the HDF5 file container. These libraries will need to be updates to enable reading and writing of of new data types.

1.3.1. Common features

Support for new data types should be added. This includes adding

(appending) data and reading data. The reading and writing should be possible through either an interface that accesses a specific type of data (e.g., MRI acquisitions) or an interface where the next chronological item is accessed. This will involve implementing a base class for all data types. All data types should also have serialization and deserialization functions needed for network communication (see below).

1.3.2. C++ library

The current C++ library is actually a C library with a C++ wrapper. The underlaying C layer should be removed to simplify the library. Data types should be added as described above.

1.3.3. Python library

The library needs support for new data types and it should be compatible with both Python 2.7 and Python 3.

1.3.4. Matlab library

The library needs support for new data types as described above.

Task 1.4. Unit testing Unit testing should be implemented for all support library functionality described above.

Task 1.5. Standalone applications The software libraries in C++, Matlab, and Python should include examples of standalone applications that a) demonstrate how the libraries are to be used and

b) generate useful test data sets. Such applications are available in the current version of ISMRMRD but they should be extended to include new features and illustrate some of the new data types. Specifically this will include the following sub tasks.

1.5.1. Data Generator

The current version of ISMRMRD includes a synthetic data generator (simulator), which generates datasets based on the Shepp-Logan phantom. This simulator includes generation of multiple receiver coils, etc. The data simulator should be updated and enhanced to include a) basic Cartesian MRI acquisition simulation with multiple noise and adjustable Gaussian noise levels, b) non-Cartesian (spiral and radial) data generation with multiple coils and adjustable Gausian noise levels, c) Cartesian MRI simulation with motion and simulated ECG signal waveforms, d) Cartesian and spiral MRI simulation with pulse sequence (RF, gradients, and ADC) waveforms.

1.5.2. Reconstruction programs

The simulated acquisitions listed in 1.5.1 should have associated image reconstruction programs in C++, Matlab, and Python. For the cases a) and b) the reconstruction tools should simply produce images. In the case with motion and simulated ECG waveform (c), the images should have the ECG waveform with an indicator for current position in the ECG waveform imprinted in the images. In the case with pulse sequence waveforms, the k-space locations for the samples should be calculated from the gradient waveforms and ADC timing and reconstructed accordingly.

1.5.3. Conversion tool from ISMRMRD version 1.x to 2.x

There should be a tool to convert existing (1.x) data files to the new data format.

Task 1.6. ISMRMRD Network Communication Protocol The contractor shall define a multi-layered network communication protocol for transmission of ISMRMRD 2.x. The lowest level of this communication protocol will consist of framed packages that start with a package size followed by a clear indication that the package contains an ISMRMRD data type and which data type it is. The second level will consist of the description of the serialized formats of all defined data types. The third layer shall include appropriate communication sequences for initiating a connection between applications that exchange ISMRMRD data, this includes any handshakes, etc. The communication protocol definitions shall include details such as endianness.

Task 1.7. Client/Server applications The software libraries in C++, Matlab, and Python should also include example client/server applications that achieve the reconstruction results defined in 1.5.2 by transmitting the simulated data sets from a client to a server (which will perform the reconstruction). The client should receive the reconstructed images and write them to a new ISMRMRD data set. There should be three (3) clients (one in each language) and three (3) servers and there should be complete interoperability between clients and servers written in different languages (C++, Matlab, Python). Specifically, any client should be able to send data to any server and the reconstruction results should be the same.

Task 1.8. Integration Tests The software packages should include an automated system for running integration tests that verify that all the applications in 1.5, 1.6, and 1.7 produce the expected results.

Task 1.9. Cross Platform Conformance All described software components should build (if applicable) and run in Windows, Linux, and Mac operating system environments.

Task 1.10. Vendor Specific Data Converters The ISMRMRD project currently includes data converters for MRI raw data acquired with Siemens, Philips, GE, and Bruker scanners. These converters should be updated such that they produce the updated ISMRMRD format.

Task 1.12. Documentation All software components (libraries and standalone applications) shall have documentation, which includes a general description of the function of the software components and in the case of libraries, API documentation should be automatically generated (e.g., with Doxygen or similar tools) based on inline code comments. Documentation shall also include build and installation instructions for Windows, Linux and Mac operating system environments.

Task 2. Gadgetron Improvements The Gadgetron (https://gadgetron.github.io) is an image reconstruction framework that currently supports a wide range of clinical and research application. This framework will need to be updated to be compatible with the new ISMRMRD format (as described above). In addition to these data standard conformance modifications, the framework also needs modernization of certain parts of the reconstruction framework.

Task 2.1. ISMRMRD 2.x compatibility The Gadgetron includes a modular deserialization/serialization interface that allows it to receive and send arbitrary data types as long as appropriate readers/writers (plugins) are available. In order to make the Gadgetron compatible with the modified ISMRMRD format, the following changes will be required:

2.1.1. Framed package compatibility

The current data transmission protocol used by the Gadgetron does not include framed packages. The contractor shall implement appropriate modifications for support of framed packages. After these modifications the Gadgetron shall only operate with framed packages.

2.1.2. Readers/Writers ISMRMRD data types

All ISMRMRD types shall have readers (deserialization modules) and writers (serialization) modules. These modules shall be implemented with the serialization and deserialization functions implemented in Task

1. The Gadgetron uses a system of slots to assign readers and writers to specific data types. This system will need adjustments to take into account how the data types are specified the framed packages. This specification includes both an ISMRMRD indicator and a data type ID.

The combined ID of the package should be used as slot ID.

Task 2.2. CPU/GPU interoperability The Gadgetron contains a library of numerical optimizers/solvers with associated operators and support routines. Many of these are implemented on the GPU. The contractor shall ensure that all optimizers (solvers), operators, and support routines that are implemented on the GPU also have CPU implementations. It is desired that the Gadgetron be able to select the CPU implementations as fallback implementations on systems where a GPU is not available or where GPU resources or specifications may be inadequate.

Task 2.3. Homogenization of reconstruction pipelines The reconstruction pipelines (series of processing steps defined as Gadgets) included with the Gadgetron have been developed over a number of years during which time conventions regarding intermediate data types (aggregate data structures) have changed. As MRI data passes down the reconstruction pipeline, it is modified and transformed and at various stages the data is accumulated (aggregated) into collections of data elements that are then processed as single units. These collections of data elements form common data structures that are inputs to various algorithm components included in the Gadgetron toolboxes, but the lack of consistency of the data types in different parts of the code creates non-uniform interfaces. The latest pipelines are designed such that data is collected first into unordered collections of data elements known as Buckets which are subsequently transformed into array representations of MRI data space (k-space arrays). These ordered array structures are seven dimensional structures known as Buffers. In practice the ordered data structures may consist of multiple Buffer structures hold actual imaging data and calibration data (for parallel MRI calibration). The contractor shall work through all reconstruction pipelines and ensure that a uniform set of aggregate data structures are used. Specifically, this task will include rewrite several Gadgets responsible for collecting data for GPU processing such that they use the Bucket/Buffer architecture outlined above. The Generic_Cartesian_Grappa.xml reconstruction chain in the current version of the Gadgetron is an example of a reconstruction chain that adheres to this uniform pipeline architecture.

Task 2.4. Python representations of all ISMRMRD data types and aggregate types The Gadgetron supports Gadgets implemented in Python. The Python integration relies on Python representations of data types that are exchanged between C++ and Python. This includes some data conversion functionality. The contractor shall ensure that all ISMRMRD data types have such Python representations and shall implement Python representations of aggregate data types (Buckets and Buffers, see above) to allow exchange of these data types between C++ and Python components.

Task 2.5. Gadgetron implementation of ISMRMRD client/server test cases The Gadgetron includes an ISMRMRD client. This client shall be updated with any modifications needed in connection with the new ISMRMRD data types and Gadgetron implementations of the reconstructions in the ISMRMRD client/server examples (Task 1.7) shall be implemented such that the Gadgetron ISMRMRD client and the Gadgetron itself can serve as client or server respectively in the client/server tests defined for ISMRMRD.

Task 2.6. Updates of Gadgetron integration tests In connection with the adjustments described above, the Gadgetron integration test libraries will need to be updated such that all the reference reconstruction results have ISMRMRD 2.x format. The contractor shall also add new integration tests cases to cover functionality added by the modifications above. In particular, the reconstruction chains described in Task 2.5 (the ISMRMRD client/server examples) shall be included as test cases.

2.2. Reporting Requirements and Deliverables:

The contractor shall submit monthly progress reports to the Contracting Officer’s Representative (COR) during the course of the contract. Each report shall be submitted in PDF format to the COR by the seventh calendar day of the month. The reports shall include a summation of the work performed associated with the task(s) within the reporting period as well as a revised timeline/milestone schedule for the remainder of the project.

2.3. Period of Performance

The period of performance of this contract will be two (2) years from the date of contract award.

This will consist of one (1) base year and one (1) option year.