Slicer NRRD support

I am having trouble loading some NRRD files into slicer. Here is a link to three images: ok.nhdr loads fine, ok2.nii.gz loads fine, bad.nhdr does not load in Slicer 4.10.2 on MacOS. All of these images load fine in MRIcroGL and ImageJ.

The Slicer error log reports:

UpdateFromFile: Unsupported number of components (only 1 allowed): 2

ReadData: Cannot read file as a volume of type DiffusionTensorVolume[fullName = /Users/chris/Desktop/bad.nhdr]
Number of files listed in the node = 0.
File reader says it was able to read 1 files.
File reader used the archetype file name of /Users/chris/Desktop/bad.nhdr [reader 0th file name = /Users/chris/Desktop/bad.nhdr]
FileFormatError

ReadData: MRMLVolumeNode does not match file kind

ReadData: Failed to instantiate a file reader

ReadData: Cannot read file as a volume of type Volume[fullName = /Users/chris/Desktop/bad.nhdr]
Number of files listed in the node = 0.
File reader says it was able to read 1 files.
File reader used the archetype file name of /Users/chris/Desktop/bad.nhdr [reader 0th file name = /Users/chris/Desktop/bad.nhdr]
FileFormatError

It seems that the volume has 2 components. Slicer core can load single-component volumes as scalar, 3-component volumes as RGB, 9-component volumes as tensor. You can load 2-component volume as a volume sequence: install Sequences extension and in “Add data” dialog choose “Volume sequence” in Description column.

@lassoan you are correct. Slicer does not load 2-volume NRRD files, though it can load 2-volume NIfTI files. I can understand this for tools that use a photometric interpretation for data - e.g. if there is only one component treat it as scalar grayscale, if there are two component generate and error, if there are three component treat it as Red/Green/Blue. However, it does not make sense when each channel is treated as a scalar volume - which is how Slicer treats 1 volume and 3 volume NRRDs. I do think the current behavior will frustrate users, and the error message does not do a good job of clarifying the solution.

Here is a link to the same volume saved as NIfTI (.hdr/.img) and NRRD (.nhdr/.img; note image files are shared by both NIfTI and NRRD so literally identical data), with separate images storing 1, 2 or 3 volumes. As you predicted, slicer loads all NIfTI images correctly but fails for the two volume NRRD.

@Chris_Rorden what’s the use case for a 2 component nrrd? Is it a time series or something else? How would you expect it to load?

1 Like

It should be treated just like the 1 and 3 volume NRRD files in slicer: as a scalar image (in this case with two volumes), and the same as the 2 volume NIfTI. At the moment, dragging and dropping a multi-volume NIfTI or NRRD just displays the first volume of the series, but it seems odd that a 2-volume NRRD is treated differently than the 1 and 3-volume files. Ideally, it would be great if all the volumes are loaded and the user could choose which volume is displayed, just like ImageJ, FSLeyes and MRIcroGL. Rapidly cycling through raw DWI and fMRI volumes is useful for detecting head movements, reconstructor errors, etc.

Here is a link to minimal dataset of 1/2/3 volume image in NRRD and NHDR.

1 Like

I guess the reason is that some file formats don’t give a clue how to interpret dimensions (for example, .mhd) and so ITK and/or Slicer was not prepared to use descriptions in file formats that do specify dimensions. This could be improved for sure, probably by promoting Volume Sequences a bit more. We plan to bundle Sequences in default Slicer installations in Slicer5 and we could probably load non-scalar/RGB/tensor volumes as volume sequences.

I think this is all available if you load the file as a volume sequence. You can go prev/next frame, play/pause using toolbar buttons or pressing Ctrl+Shift+Left arrow/Right arrow. You can browse multiple sequences separately or in sync, perform motion compensation (using Sequence Registration), etc.

Perhaps ITK and Slicer could be enhanced to leverage the kinds field.

As an aside, dcm2niix always converts DICOM to NRRD kinds: space space space list, but perhaps it should be enhanced so that list is used for DWI data and time is used for ASL, fMRI, etc. This would hint that one could interpolate volumes along time, where this would not be appropriate for sequentially stored DWI directions. Perhaps @tbillah could comment on this proposal.

1 Like

Sequences extension uses kinds field, too, but always writes “list” for the vector dimension (and uses labels and units fields to store actual name and unit of the dimension), as we did not want to deviate from what ITK and most other packages do. If tools start to use kinds field properly then we can update Slicer to do the same.

I agree with the goal of improving the semantics of the files so that software can automatically do something more useful. There are definitely cases in the Slicer (and ITK) codebase where some kind of heuristic is the only option.

Using the kinds field in nrrd is good and it would be great to see it respected more broadly.

On the other extreme we (as in me, @fedorov, etc) are also continuing to work on encoding complex data types in dicom to be even more explicit about the meanings. For example, parametric maps can explicitly encode the values at each pixel, but also the quantity measured and the units in which it is measured. They can also include references to the data from which they were derived in addition to the usual patient/study metadata.

Slicer can also load NRRD with any number of components, including 2, as a multivolume, but the header must match the (un-documented) expectations of the multivolume reader. It is mostly intended to read back multivolumes saved by Slicer.

fedorov cans you provide a sample header with these undocumented features or briefly discuss them. That way I could update dcm2niix to support this. Below is the current NRRD header created for a simple DICOM 2-volume dataset.

NRRD0005
# Complete NRRD file format specification at:
# http://teem.sourceforge.net/nrrd/format.html
# dcm2niix v1.0.20190808 NRRD export transforms by Tashrif Billah
type: int16
dimension: 4
space: right-anterior-superior
sizes: 64 64 35 2
thicknesses:  NaN  NaN 3 NaN
endian: little
encoding: gzip
data file: 2vol_ax_asc_35sl_20140310133834_6.raw.gz
space units: "mm" "mm" "mm"
space origin: (104,-58.6843,-84.798)
space directions: (-3.25,3.25e-16,0) (3.25e-16,3.23099,0.350998) (-3.88798e-17,-0.388798,3.57894) none
centerings: cell cell cell ???
kinds: space space space list
DICOM_0008_0060_Modality:=MR
DICOM_0008_0070_Manufacturer:=SIEMENS
DICOM_0008_1090_ManufacturerModelName:=TrioTim
DICOM_0018_0022_ScanOptions:=FS
DICOM_0018_0023_MRAcquisitionType:=2D
DICOM_0018_0080_RepetitionTime:=3000
DICOM_0018_0081_EchoTime:=30
DICOM_0018_0083_NumberOfAverages:=1
DICOM_0018_0087_MagneticFieldStrength:=3
DICOM_0018_1020_SoftwareVersions:=syngo_MR_B17
DICOM_0018_1314_FlipAngle:=76

Slicer should not require custom tags to read a 4D volume, so I would not recommend to modify dcm2niix.

The plan is that Sequences will replace multivolume infrastructure in Slicer but it will require at least several months to bring Sequences into Slicer core. If we need an solution sooner then we can remove the special tag requirements in multivolume importer.

@lassoan I don’t know what you mean by the first part (those custom tags describe semantics of the volume, and replicate some of the attributes from the DICOM header - I am not sure there is a “standard” NRRD or NIfTI way to do this, since dcm2niix saves some of those into a separate JSON file for BIDS), but I agree it is not a good idea to modify dcm2niix to write custom attributes that are only readable by the multivolume reader in Slicer.

There are standard fields for axis kind, measurement unit, etc., so multi-volume modules should not require presence of any custom fields to load a 4D volume as multi-volume node.

I agree that additional custom fields would be useful (e.g., to store frame time vector) and agreeing in a standard list of fields between dcm2niix and Slicer could simplify DICOM import. However, I’m not sure how to implement this. Should this be a small private business between dcm2niix and Slicer (and hope that others will follow) or we should try to involve medical image computing and scipy community? There are many people who would be interested in consolidation in file formats and tools (see for example this discussion and this), but nobody has promised to allocate resources or offer to write a grant application.

@fedorov, can improving research file formats and tools be part of your NCI Imaging Data Commons project, or it can only use DICOM?

No, this is not in the scope. We may need to revisit it, who knows, but at least our declared plan is to use DICOM for all data, both images and image-derived.

Note that multivolumes, as produced by the multivolume plugin, will often have irregular spacing in the 4th dimension (e.g., for various types of MR acquisitions). Is that something that is supported by NRRD?

Sadly, nrrd cannot store variable spaced data along any dimension in a standard way. You can have origin and spacing (start time and time between frames) but not arbitrary value for each frame.

Regardless, we should still be able to read/write nrrd files that does not have custom timestamps. If custom timestamps are found in a field then we could use those.

Nifti seems to have some slice_times field, but I’m not sure how standard it is.

@lassoan the NIfTI format was designed for MRI scans prior to the advent of multi-band. It describes the relative timing of the 2D slices that are stacked to form a 4D volume.

  • The timing between volumes is always fixed (TR = pixdim[4]). This makes sense for MRI, as variable TR impacts the image contrast. However, I think @fedorov would like to describe variable times between volumes for the CT modality. This is a useful feature, but not supported by NIfTI.
  • The NIfTI format only describes ascending, descending, odd-first interleaved and even-first interleaved patterns of 2D slices. This is sufficient to allow slice time correction for single-band MRI. However, it can not describe multi-band sequences, e.g. with a 32-slice volume and MB=4, a total of 8 readouts are performed each with 4 aliased slices (though one could argue that MB allows very short TRs, so the slice-timing error is very small allowing the temporal derivative to soak up variability and allowing one to ignore slice time correction).
  • Tools like dcm2niix can create a BIDS format JSON text file with the SliceTiming tag that independently does describe the time of each slice in a volume. Therefore, this can describe multi band. However, it still does not provide a mechanism to describe variable time between the 3D volumes that compose a 4D series.

Since the NIfTI format can not support variable timing between volumes, perhaps NRRD could be formally extended. It appears the Slicer has informally introduced the NRRD tag MultiVolume.FrameLabels: MultiVolume.FrameLabels:=0.0,4270.0,8541.0,12812.0,.... If some scheme like this could be adopted as part of the NRRD specification, it could solve @fedorov’s issue (though I realize he is also advocating that DICOM might be a flexible format to handle this situation).

By the way, the full description of NIfTI slice timing support is embedded in the NIfTI header:

slice_duration = If this is positive, AND if slice_dim is nonzero,
                 indicates the amount of time used to acquire 1 slice.
                 slice_duration*dim[slice_dim] can be less than pixdim[4]
                 with a clustered acquisition method, for example.

slice_code = If this is nonzero, AND if slice_dim is nonzero, AND
             if slice_duration is positive, indicates the timing
             pattern of the slice acquisition.  The following codes
             are defined:
               NIFTI_SLICE_SEQ_INC  == sequential increasing
               NIFTI_SLICE_SEQ_DEC  == sequential decreasing
               NIFTI_SLICE_ALT_INC  == alternating increasing
               NIFTI_SLICE_ALT_DEC  == alternating decreasing
               NIFTI_SLICE_ALT_INC2 == alternating increasing #2
               NIFTI_SLICE_ALT_DEC2 == alternating decreasing #2
{ slice_start } = Indicates the start and end of the slice acquisition
{ slice_end   } = pattern, when slice_code is nonzero.  These values
                are present to allow for the possible addition of
                "padded" slices at either end of the volume, which
                don't fit into the slice timing pattern.  If there
                are no padding slices, then slice_start=0 and
                slice_end=dim[slice_dim]-1 are the correct values.
                For these values to be meaningful, slice_start must
                be non-negative and slice_end must be greater than
                slice_start.  Otherwise, they should be ignored.

Personally, I think different formats fill different niches: NIfTI/NRRD are very simple and fast to parse. They lend themselves to datascientists and developers who want to tools that can quickly read and write tools with simple I/O. The DICOM format is very flexible, but extremely complex, compounded by the fact that different vendors have interpreted the standard differently. Ideally, it would be great if the DICOM standard could describe a “core” usage, similar to how one can use the legacy OpenGL or the streamlined “ES” and “Core” specifications. Since the “Core” features are a subset of the full specification, systems would be backward compatible. However, new enhanced files that followed the core pattern would be much faster to parse (in particular if group lengths were mandatory). Until such a time, DICOM is extremely flexible but unwieldy for many projects. While I respect @Fedorov and David Clunie’s feeling that DICOM should be THE medical imaging format, I think the success of NIfTI and NRRD reflect a Darwinian selection. Different species have developed that are each adapted to their niche. If we really want to see wider adoption of DICOM in tools, we need to explore how DICOM can evolve to succeed in more environments (simplified core specification, core validator, etc.).

2 Likes

The motivation for multi-volume was actually just various flavors of MRI in the context of prostate imaging. DCE timing interval is fixed, but other applications for prostate imaging include various T1- and T2-mapping sequences (variable FA or TR), and DW trace images with unequally spaced b-values (e.g., see Evaluation of fitting models for prostate tissue characterization using extended-range b-factor diffusion-weighted imaging - PubMed). In multivolumes, from the start, the forth dimension was not intended to always correspond to time.

I agree with many points, but I think once you want to step out of the boundaries defined by those formats, and introduce extra complexity, in particular - handling metadata (like the issue we are discussing here), the answer is not trivial. I am not aware of examples that use NRRD or NIfTI (or BIDS, for that matter) for large-scale data aggregation and metadata management.

The question becomes do we want to invest the time to further extend those formats, or explore how those situations can be handled in DICOM. Yes, vendors have different implementations and interpretations of DICOM, but why not try to establish best practices and open source DICOM conversion tools that we can iteratively refine? We have just a handful of established tools that normalize vendor-specific DICOM and output NRRD/NIfTI - why not explore how those tools can output multiframe DICOM in addition to the research format they output right now?

I agree at this point it is hard to make the case for using DICOM to solve this specific problem, since there are no converters that could “just do it”. But I would much rather explore how this problem can be solved using DICOM. And if we have more people who believe this approach is worth the time investment, we would much sooner be able to decide how suitable multiframe DICOM is for those tasks, how much effort it will take to implement it, and how to proceed with refining the standard along the lines you mentioned, Chris.

2 Likes

@fedorov I agree on all points. Tools that convert vendor-specific DICOMs to general NIfTI/NRRD could easily be adapted to generate a clean and simple DICOM. However, I think this effort would work best if there was a clear idea of what that flavor of DICOM would look like. People like you and David Clunie seem well positioned to do this. A clean specification and validation tool would allow developers of different tools to tune their conversions.

2 Likes