Open Nifti in 3D Slicer

Support
1

I usually converted the image data from our small animal PET scanner into Nifti to be able to open it in slicer. In the newer versions we have problems and get an error message. Neither newly converted data nor Nifti files that were previously already opened in slicer still work.

“Error occured while loading selected files.” Details: “Error: Loading (path+file name) - load failed.”

In version 4.10.xx it still works, in all versions 5.0x it does not work. I am using the windows-version.
Is there any further information needed to reproduce the problem?

I am grateful for any advice on how I should proceed.

Thank you in advance,
Gordon

2

Yes, please provide a sample file (via dropbox, google drive etc) that demonstrates the issue.

Also note that we routinely tell people to avoid the nifti format if you can since it is very common to get ill-formed files depending on the software that generates them.

3

So far, the Nifti format had worked well for us and we had not noticed any problems. We will now also check, if we can convert the image data to some other file format.

For test, please feel free the following data:

I could open this file in 3D Slicer versions 4.10.2 and 4.11.2020930 but not in 5.0.3 or 5.1.0-2022-07-22.

4

I’ve debugged this issue and the problem according to the NIFTI reader in ITK is that ITK only supports orthonormal direction cosines. No orthonormal definition found!. I would recommend to report this issue on the ITK forum and in the meantime switch to NRRD.

You have been really lucky that this is the first time you ran into problem because of NIFTI. NIFTI is really just so unnecessarily complicated and fragile file format that everyone should stay away from it (except neuroimaging researchers, because they have built a lot of tools around it over the years that it would be hard to drop it; and there are also some useful features for brain imaging).

5

Just for me to understand again. The Nifti files are opened in Slicer via ITK?
And ITK has changed from Slicer version 4 to version 5?

ITK-SNAP 3.8 can open the file with 16-bit precision, but not the original 32-bit.

Do you have any recommendation for converting image file from preclinical PET to NRRD? I have used Vinci for conversion to Nifti. The software does not convert to NRRD.

VINCI | About (mpg.de)

Original PET-Data:

I would be very grateful for any further advice.

6

Background

I respectfully disagree with @lassoan’s comments regarding NIfTI being fragile. From my perspective, the NIfTI format is very simple (just 348 bytes) and explicit, but it is tuned to a specific domain. Relative to other formats like DICOM, NRRD and NIfTI are very similar. One main difference between the two is that to support NRRD you need a very good reader to parse the various spatial frames of reference. On the other hand, NIfTI uses only a single spatial coordinate system so reading is simple, but the writing of NIfTI is challenging as you must use the NIfTI frame of reference.

The biggest issue many have with NIfTI is that it can include two different spatial transforms - the QForm and the SForm. The qform_code and sform_code allow you to understand what they refer to. While usage can vary, the original rationale is that the Q-Form code could map the image alignment to the scanner’s acquisition space, while the S-Form can map the image to a standard space where the participants brain is aligned to a typical brain size, rotation and origin (with a full affine 12 degrees of freedom: 3 axes rotation, 3 axes translation, 3 axes zoom and 3 axes shear). It is particularly unfortunate that the NIfTI creation specified that the Q Form uses a quaternion (which only has 9 degrees of freedom) while the S Form uses a matrix (with 12 DoF). This does complicate support for the format. Also, since the SForm supports shears, where many image viewers require are unable to support rhomboidal images.

The Insight ToolKit (ITK) does not support rhomboidal images. While this may have changed, early versions of ITK always gave precedence to the QForm (presumably because the constrained QForm can not represent rhomboidal images), while most other tools chose between the SForm and QForm by selecting the higher qform_code versus sform_code (and giving precedence to the SForm in the case of ties). This meant that the starting estimates for ITK based tools like ANTS were often very different from other NIfTI tools like SPM and FSL.

For these reasons, most tools today create NIfTI images where the SForm and QForm are identical. Rhomboidal images are virtually never seen in MRI (though they exist with CT scans acquired with gantry tilt), so most tools limit NIfTI images to 9 degrees of freedom.

Problem with your image

Long story short, I think the core issue with your image is that it fails to specify either a sform_code or a qform_code, and the two spatial representations are mutually exclusive:

fslhd ChickenEgg-PET-Data.nii
...
qform_name	Unknown
qform_code	0
qto_xyz:1	0.865759 0.000000 0.000000 0.000000 
qto_xyz:2	0.000000 0.865759 0.000000 0.000000 
qto_xyz:3	0.000000 0.000000 0.796000 0.000000 
qto_xyz:4	0.000000 0.000000 0.000000 1.000000 
qform_xorient	Left-to-Right
qform_yorient	Posterior-to-Anterior
qform_zorient	Inferior-to-Superior
sform_name	Unknown
sform_code	2
sto_xyz:1	1.000000 0.000000 0.000000 -54.975697 
sto_xyz:2	0.000000 1.000000 0.000000 -54.975697 
sto_xyz:3	0.000000 0.000000 1.000000 -37.411999 
sto_xyz:4	0.000000 0.000000 0.000000 1.000000

Solution

The easiest fix is would be to make sure the SForm and QForm store the same representation (e.g. the QForm looks plausible but adding a translation to set the origin is required) and set each to the QForm_code and SForm_code to scanner_anat.

Longer term, it looks like your data was acquired on a CTI Concorde model 2501. I have never seen the text format for the header you provided. I would see if you can either export the data from the scanner as DICOM or ECAT. You can use dcm2niix to convert those to clean NIfTI or NRRD datasets. dcm2niix can be run from the command line, but it is also available as a slice extension. You might also want to look at PET2BIDS which is a wrapper from dcm2niix that helps specify a lot of the PET details that are useful for curated archival datasets.

2 Likes
7

The Nifti files are opened in Slicer via ITK?

Yes

And ITK has changed from Slicer version 4 to version 5?

Yes. BTW, the error message appears in the commit ENH: Prefer to use sform over qform when both are set.
I have checked, older versions of my app (uses ITK IO for Nifti), like Slicer, can open the nii file:
Spacing: [0.865759, 0.865759, 0.796]
Origin: [54.9757, 54.9757, -37.412]
Direction:
-1 0 0
0 -1 0
0 0 1

(in LPS space)

New version fails, of course.

BTW,

$nifti_tool -check_hdr -infiles ChickenEgg-PET-Data.nii
said
header IS GOOD for file ChickenEgg-PET-Data.nii

The header from nifti_tool:

…
pixdim 76 8 0.0 0.865759 0.865759 0.796 0.0 0.0 0.0 0.0
…
qform_code 344 1 0
sform_code 348 1 2
quatern_b 352 1 0.0
quatern_c 360 1 0.0
quatern_d 368 1 0.0
qoffset_x 376 1 0.0
qoffset_y 384 1 0.0
qoffset_z 392 1 0.0
srow_x 400 4 1.0 0.0 0.0 -54.975697
srow_y 432 4 0.0 1.0 0.0 -54.975697
srow_z 464 4 0.0 0.0 1.0 -37.411999
…

Strange why it fails, specially if sform should be preferred over qform. Probably authors of ITK’s Nifti IO should take a look.

The Analyse7.5(?) img file worked with ITK 4.13 and fails now too.

2 Likes
8

Thank you very much for the detailed answers.

About the data: It is image data from a small animal PET scanner FOCUS 120 from Concorde Microsystem Inc, later Siemens. I have only been able to open the image data with ITK Snap 3.8, Amide and Vinci. ITK Snap 3.8 brings the mentioned error message concerning the 16-bit precision.
For Slicer I had therefore converted the data so far by means of Vinci into Nifti.

At least in Vinci another format is given under “Analysis Image”, I am not sure if Analysis7.5 corresponds to the format of the FOCUS 120.

So as far as I understand the answers correctly, the reason is most likely a changed I/O at the ITK.

Thanks again, then I’ll see if I can realize the other conversions!

9

@Chris_Rorden and @issakomi Thanks a lot for sharing your valuable insights!

By “fragile” I meant that if you use NIFTI then it is easy to end up with invalid or ambiguous data because there are so many things that can go wrong.

This seems to be due to inherent flaws of this file format, because the same software developers that struggle to read/write NIFTI correctly have no trouble implementing correct readers and writers for other research file formats with similar capabilities, such as NRRD or MetaIO.

1 Like
10

Thanks everyone for the informative discussion.

If this ChickenEgg file is well formed but not readable by the current ITK then for sure the ITK community should be informed and perhaps this can be addressed in a future version (if no one has done it yet, starting a thread at https://discourse.itk.org/ would be a good place to start, as @lassoan suggested).

For Slicer, we can often represent images with more generality than ITK, for example by allowing gantry tilt, missing slices, etc. We typically focus on DICOM for this but if someone wanted to do it it would be possible to introduce options in the nifti reader that would handle these variants.

For example, here’s a work-in-progress module for a custom reader for a common diffusion nifti convention (where there are .bval and .bvec files in the directory with the .nii.gz) that users can optionally select instead of the default ITK-based reader). Someone could do a similar plugin using something like nibabel in python to read the nifti with options to handle the header variants.

11

@lassoan perhaps you can provide examples? From my perspective, NIfTI is extremely explicit and simple. While you can create malformed NIfTIs, the same is true of NRRD. For both, there are obvious correct ways of doing things. It is unfortunate that the NIfTI specification used both quaternions and matrices, which requires a reader to support both types (though a writer can just choose one). For full disclosure, I advocated for not including the QForm when NIfTI was created. I do think the ITK standards implementation of NIfTI was completely different from others. I did ask the ITK team to try to use the same approach as SPM/FSL/etc many, many years ago and was told they would not change things due to fear of unintended consequences. I am glad to know that the latest version of ITK behaves like other tools.

I do think NRRD is nice because you can use a NRRD header to describe an image saved in a different format, adapting to the original formats spatial frame of reference.

I think both NRRD and NIfTI fill a niche, but it seems that NIfTI has gained much broader adoption in the field of brain imaging.

12

This very topic is an example. But there are many more examples that indicate that for many users and developers NIFTI is too complex and prone to misinterpretation:

13

I have looked a little closer at the issue with IO, the problem is at this line

if (itk::Math::abs(this->m_NiftiImage->dx - scale[0]) > large_value_tolerance)

dx is 0.865759 and scale[0] is 1, the variable sform_decomposable_without_skew is set to false here and code runs up to the exception.

Looks like ITK IO expects a matrix with respect to spacing here, something like
IndexToPointMatrix

  srow_x   0.865759 0.0      0.0      -54.975697
  srow_y   0.0      0.865759 0.0      -54.975697
  srow_z   0.0      0.0      0.796    -37.411999

and not

  srow_x    1.0   0.0   0.0      -54.975697
  srow_y    0.0   1.0    0.0     -54.975697
  srow_z    0.0   0.0    1.0     -37.411999

I don’t know is it correct behavior or not, just FYI.

Edit:
old version worked fine, BTW, spacing and orientation were correct.

14

@lassoan I think a lot of these simply reflect the popularity of NIfTI versus NRRD. It is certainly possible to save a NRRD with shears using space directions. I think most NRRD creation tools simply enforce orthogonal volumes that match ITKs desires.

The comments about limited hardcoded meta data is a strength from the perspective of simplicity. NRRD is a bit of the Wild West in terms of defining meta data, so when I tried to make dcm2niix NRRD export compatible with Slicer I found that there really is not standard way of preserving much of the meta data even if you can pack it all in a human readable form. As noted by others in the comments you link, many of these revolve around ITK’s implementation. As I noted earlier, I tried to work with them well over a decade ago to resolve incompatibilities to no avail. As @ihnorton noted in one of those posts, ITK uses LPS, while NIfTI uses RAS. However, I do not know enough about slicer to understand why this is not handle my reorienting each image to the desired orientation and storing the residual rotation matrix. That is the approach NiiVue uses - with each volume always approximately RAS and storing a residual as well as an inverse transform.

Again, I do not think NIfTI is perfect. The inclusion of quaternions is very alien to most users, and it does require every tool to handle (typically by converting them to a matrix and a potentially lossy transform of a matrix). However, if one wants to be the advocate for quaternions you could see their inability to store shears as a strength. It works well with tools like ITK that can not abide by rhomboidal volumes and prevents shears from accumulating doing to rounding errors.

15

@issakomi it looks like you have done nice detective work. The current ITK code does not seem to have a robust detection of skew, and such a test would fail for both a NRRD and NIfTI that included a sheared matrix. I think the reliable way to detect this would be to leverage the fact that quaternions can not retain shears. So given a matrix that you want to test, convert it to a quaternion and convert that quaternion to a matrix. This will be identical to the input if there is no shear. You can compare the direction of the row, column and slice vectors to provide a human-readable tolerance for shear (e.g. difference in degrees).

ITK already includes these transforms in the NIfTI reading: nifti_mat44_to_quatern and nifti_quatern_to_mat44 if one wants to do this completely in the NIfTI reader. On the other hand, a more general solution that will work for NRRD files could use the vnl_matrix_fixed and vnl_quaternion functions.

16

A simpler approach is to simply measure the angles for the row versus column, row versus slice and column versus slice. These should all be 90 degrees.

Here is a JavaScript implementation

17

I’ve thought about this, too, but many problems simply cannot occur with other formats, because it is much harder to create ambiguous or invalid files.

For example, other research image file formats use origin, spacing, axis direction vectors (not quaternion) to specify image geometry and explicitly allow non-orthogonal image axes. ITK image class can store images with non-orthogonal axes and ITK readers, writers, and resampling filters support such images. So, there is a simple, unambiguous process to handle sheared images in ITK.

I agree. But ITK developers are smart people with lots of domain knowledge and connections to the neuroimaging community, so if they cannot implement a robust file reader/writer for NIfTI in 10-20 years then there must be something inherently wrong with this format.

The RAS/LPS convention is not an issue. ITK always loads images into LPS coordinate system, performing any necessary conversions. The image orientation problems are usually due to flip around a single axis.

I think the main problems are the usage quaternion, redundancy of sform/qform, and inconsistent behavior of various implementations.

18

Not at all my perspective, NIfTI is simple and supported robustly by a huge number of tools, including major ones like SPM, FSL and AFNI. NIfTI was the lowest common denominator that allowed these tools to work robustly. The ITK implementation is the odd one out. Early ITK implementations gave precedence to the QForm regardless of the Q/SForm codes and were divergent from all other implementations and the NIfTI specification. These choices caused ITK based tools to behave oddly. For many years, the ITK team refused to conform to the standard or other tools, for the sake of backward compatibility. I agree that the ITK team is talented engineers, but they made a willful choice of maintaining the behavior of the software to preserve behavior with previous versions of ITK, regardless of the behavior of all the other tools. The SPM team even changed their NIfTI writer to make the SForm and QForm the same (to the extent possible) to support ITK based tools. I am glad that the ITK team now supports the specification, and I do hope this bug will lead to consistent behavior that is similar to other tools and reduce the number of issues the Slicer developers and users are experiencing.

19

ITK should be changed so that it works consistently with the major neuro software packages, which should reduce friction between ITK-based tools and SPM/FSL/AFNI. If there are any remaining issues in the current ITK implementation then please ask on the ITK developers to address it. I and other Slicer community members will support this any way we can. If there is anything to change how ITK NIfTI IO is used in Slicer then let us know.

Still, this does not address the world outside the neuroimaging community. I see many incorrect/ambiguous NIfTI images out there and bad ones are still being created - either because of incorrect writer implementations or misuse by people who just want to save their image and don’t want to learn about sform/qform. One solution would be to simplify NIfTI (e.g., by removing qform), but since the neuroimaging community does not seem to have any problem with it, this will probably not happen. That’s why I keep discouraging use of NIfTI for all users who don’t need it and don’t want to spend time with understanding it (e.g., anyone outside the neuroimaging field) and recommend to use a simpler format instead.

20

The file could be loaded if the block scope has been removed. From the sform, spacing and orientation are correct.

Spacing: [0.865759, 0.865759, 0.796]
Origin: [54.9757, 54.9757, -37.412]
Direction:
-1 0 0
0 -1 0
0 0 1

We spoke already in the past on ITK forum about sheared matrices / rhomboidal images, i am still a little pessimistic about them.

Edit:
Does your tool currently write Nrrd files with sheared matrix?