Open Nifti in 3D Slicer

The Nifti files are opened in Slicer via ITK?

Yes

And ITK has changed from Slicer version 4 to version 5?

Yes. BTW, the error message appears in the commit ENH: Prefer to use sform over qform when both are set.
I have checked, older versions of my app (uses ITK IO for Nifti), like Slicer, can open the nii file:
Spacing: [0.865759, 0.865759, 0.796]
Origin: [54.9757, 54.9757, -37.412]
Direction:
-1 0 0
0 -1 0
0 0 1

(in LPS space)

New version fails, of course.

BTW,

$nifti_tool -check_hdr -infiles ChickenEgg-PET-Data.nii
said
header IS GOOD for file ChickenEgg-PET-Data.nii

The header from nifti_tool:


pixdim 76 8 0.0 0.865759 0.865759 0.796 0.0 0.0 0.0 0.0

qform_code 344 1 0
sform_code 348 1 2
quatern_b 352 1 0.0
quatern_c 360 1 0.0
quatern_d 368 1 0.0
qoffset_x 376 1 0.0
qoffset_y 384 1 0.0
qoffset_z 392 1 0.0
srow_x 400 4 1.0 0.0 0.0 -54.975697
srow_y 432 4 0.0 1.0 0.0 -54.975697
srow_z 464 4 0.0 0.0 1.0 -37.411999

Strange why it fails, specially if sform should be preferred over qform. Probably authors of ITK’s Nifti IO should take a look.

The Analyse7.5(?) img file worked with ITK 4.13 and fails now too.

2 Likes

Thank you very much for the detailed answers.

About the data: It is image data from a small animal PET scanner FOCUS 120 from Concorde Microsystem Inc, later Siemens. I have only been able to open the image data with ITK Snap 3.8, Amide and Vinci. ITK Snap 3.8 brings the mentioned error message concerning the 16-bit precision.
For Slicer I had therefore converted the data so far by means of Vinci into Nifti.

At least in Vinci another format is given under “Analysis Image”, I am not sure if Analysis7.5 corresponds to the format of the FOCUS 120.

So as far as I understand the answers correctly, the reason is most likely a changed I/O at the ITK.

Thanks again, then I’ll see if I can realize the other conversions!

@Chris_Rorden and @issakomi Thanks a lot for sharing your valuable insights!

By “fragile” I meant that if you use NIFTI then it is easy to end up with invalid or ambiguous data because there are so many things that can go wrong.

This seems to be due to inherent flaws of this file format, because the same software developers that struggle to read/write NIFTI correctly have no trouble implementing correct readers and writers for other research file formats with similar capabilities, such as NRRD or MetaIO.

1 Like

Thanks everyone for the informative discussion.

If this ChickenEgg file is well formed but not readable by the current ITK then for sure the ITK community should be informed and perhaps this can be addressed in a future version (if no one has done it yet, starting a thread at https://discourse.itk.org/ would be a good place to start, as @lassoan suggested).

For Slicer, we can often represent images with more generality than ITK, for example by allowing gantry tilt, missing slices, etc. We typically focus on DICOM for this but if someone wanted to do it it would be possible to introduce options in the nifti reader that would handle these variants.

For example, here’s a work-in-progress module for a custom reader for a common diffusion nifti convention (where there are .bval and .bvec files in the directory with the .nii.gz) that users can optionally select instead of the default ITK-based reader). Someone could do a similar plugin using something like nibabel in python to read the nifti with options to handle the header variants.

@lassoan perhaps you can provide examples? From my perspective, NIfTI is extremely explicit and simple. While you can create malformed NIfTIs, the same is true of NRRD. For both, there are obvious correct ways of doing things. It is unfortunate that the NIfTI specification used both quaternions and matrices, which requires a reader to support both types (though a writer can just choose one). For full disclosure, I advocated for not including the QForm when NIfTI was created. I do think the ITK standards implementation of NIfTI was completely different from others. I did ask the ITK team to try to use the same approach as SPM/FSL/etc many, many years ago and was told they would not change things due to fear of unintended consequences. I am glad to know that the latest version of ITK behaves like other tools.

I do think NRRD is nice because you can use a NRRD header to describe an image saved in a different format, adapting to the original formats spatial frame of reference.

I think both NRRD and NIfTI fill a niche, but it seems that NIfTI has gained much broader adoption in the field of brain imaging.

This very topic is an example. But there are many more examples that indicate that for many users and developers NIFTI is too complex and prone to misinterpretation:

I have looked a little closer at the issue with IO, the problem is at this line

if (itk::Math::abs(this->m_NiftiImage->dx - scale[0]) > large_value_tolerance)

dx is 0.865759 and scale[0] is 1, the variable sform_decomposable_without_skew is set to false here and code runs up to the exception.

Looks like ITK IO expects a matrix with respect to spacing here, something like
IndexToPointMatrix

  srow_x   0.865759 0.0      0.0      -54.975697
  srow_y   0.0      0.865759 0.0      -54.975697
  srow_z   0.0      0.0      0.796    -37.411999

and not

  srow_x    1.0   0.0   0.0      -54.975697
  srow_y    0.0   1.0    0.0     -54.975697
  srow_z    0.0   0.0    1.0     -37.411999

I don’t know is it correct behavior or not, just FYI.

Edit:
old version worked fine, BTW, spacing and orientation were correct.

@lassoan I think a lot of these simply reflect the popularity of NIfTI versus NRRD. It is certainly possible to save a NRRD with shears using space directions. I think most NRRD creation tools simply enforce orthogonal volumes that match ITKs desires.

The comments about limited hardcoded meta data is a strength from the perspective of simplicity. NRRD is a bit of the Wild West in terms of defining meta data, so when I tried to make dcm2niix NRRD export compatible with Slicer I found that there really is not standard way of preserving much of the meta data even if you can pack it all in a human readable form. As noted by others in the comments you link, many of these revolve around ITK’s implementation. As I noted earlier, I tried to work with them well over a decade ago to resolve incompatibilities to no avail. As @ihnorton noted in one of those posts, ITK uses LPS, while NIfTI uses RAS. However, I do not know enough about slicer to understand why this is not handle my reorienting each image to the desired orientation and storing the residual rotation matrix. That is the approach NiiVue uses - with each volume always approximately RAS and storing a residual as well as an inverse transform.

Again, I do not think NIfTI is perfect. The inclusion of quaternions is very alien to most users, and it does require every tool to handle (typically by converting them to a matrix and a potentially lossy transform of a matrix). However, if one wants to be the advocate for quaternions you could see their inability to store shears as a strength. It works well with tools like ITK that can not abide by rhomboidal volumes and prevents shears from accumulating doing to rounding errors.

@issakomi it looks like you have done nice detective work. The current ITK code does not seem to have a robust detection of skew, and such a test would fail for both a NRRD and NIfTI that included a sheared matrix. I think the reliable way to detect this would be to leverage the fact that quaternions can not retain shears. So given a matrix that you want to test, convert it to a quaternion and convert that quaternion to a matrix. This will be identical to the input if there is no shear. You can compare the direction of the row, column and slice vectors to provide a human-readable tolerance for shear (e.g. difference in degrees).

ITK already includes these transforms in the NIfTI reading: nifti_mat44_to_quatern and nifti_quatern_to_mat44 if one wants to do this completely in the NIfTI reader. On the other hand, a more general solution that will work for NRRD files could use the vnl_matrix_fixed and vnl_quaternion functions.

A simpler approach is to simply measure the angles for the row versus column, row versus slice and column versus slice. These should all be 90 degrees.

Here is a JavaScript implementation

I’ve thought about this, too, but many problems simply cannot occur with other formats, because it is much harder to create ambiguous or invalid files.

For example, other research image file formats use origin, spacing, axis direction vectors (not quaternion) to specify image geometry and explicitly allow non-orthogonal image axes. ITK image class can store images with non-orthogonal axes and ITK readers, writers, and resampling filters support such images. So, there is a simple, unambiguous process to handle sheared images in ITK.

I agree. But ITK developers are smart people with lots of domain knowledge and connections to the neuroimaging community, so if they cannot implement a robust file reader/writer for NIfTI in 10-20 years then there must be something inherently wrong with this format.

The RAS/LPS convention is not an issue. ITK always loads images into LPS coordinate system, performing any necessary conversions. The image orientation problems are usually due to flip around a single axis.

I think the main problems are the usage quaternion, redundancy of sform/qform, and inconsistent behavior of various implementations.

Not at all my perspective, NIfTI is simple and supported robustly by a huge number of tools, including major ones like SPM, FSL and AFNI. NIfTI was the lowest common denominator that allowed these tools to work robustly. The ITK implementation is the odd one out. Early ITK implementations gave precedence to the QForm regardless of the Q/SForm codes and were divergent from all other implementations and the NIfTI specification. These choices caused ITK based tools to behave oddly. For many years, the ITK team refused to conform to the standard or other tools, for the sake of backward compatibility. I agree that the ITK team is talented engineers, but they made a willful choice of maintaining the behavior of the software to preserve behavior with previous versions of ITK, regardless of the behavior of all the other tools. The SPM team even changed their NIfTI writer to make the SForm and QForm the same (to the extent possible) to support ITK based tools. I am glad that the ITK team now supports the specification, and I do hope this bug will lead to consistent behavior that is similar to other tools and reduce the number of issues the Slicer developers and users are experiencing.

ITK should be changed so that it works consistently with the major neuro software packages, which should reduce friction between ITK-based tools and SPM/FSL/AFNI. If there are any remaining issues in the current ITK implementation then please ask on the ITK developers to address it. I and other Slicer community members will support this any way we can. If there is anything to change how ITK NIfTI IO is used in Slicer then let us know.

Still, this does not address the world outside the neuroimaging community. I see many incorrect/ambiguous NIfTI images out there and bad ones are still being created - either because of incorrect writer implementations or misuse by people who just want to save their image and don’t want to learn about sform/qform. One solution would be to simplify NIfTI (e.g., by removing qform), but since the neuroimaging community does not seem to have any problem with it, this will probably not happen. That’s why I keep discouraging use of NIfTI for all users who don’t need it and don’t want to spend time with understanding it (e.g., anyone outside the neuroimaging field) and recommend to use a simpler format instead.

The file could be loaded if the block scope has been removed. From the sform, spacing and orientation are correct.

Spacing: [0.865759, 0.865759, 0.796]
Origin: [54.9757, 54.9757, -37.412]
Direction:
-1 0 0
0 -1 0
0 0 1

We spoke already in the past on ITK forum about sheared matrices / rhomboidal images, i am still a little pessimistic about them.

Edit:
Does your tool currently write Nrrd files with sheared matrix?

I can answer it already. I tried dcm2niix with DICOM series (equidistant, parallel, gantry tilt) and “-e y” option, produced 2 files, one has shear matrix and another seems to be orthogonal and corrected.

The shear one looks like:

Neither Nrrd nor Nifti files with shear matrices were recognized by ITK as such. I didn’t look at Nrrd IO, but Nifti IO tried to detect it and failed.

It can be done like this, BTW, very simple (the values are from Nrrd files produced by dcm2niix):

#include <iostream>
#include <cmath>

typedef struct
{
	double x;
	double y;
	double z;
} V3;

static V3 normalize(V3 v)
{
	const double j = 1.0/(sqrt((v.x*v.x) + (v.y*v.y) + (v.z*v.z)));
	V3 vn;
	vn.x = v.x*j;
	vn.y = v.y*j;
	vn.z = v.z*j;
	return vn;
}

static double dot(V3 a, V3 b)
{
	return ((a.x*b.x) + (a.y*b.y) + (a.z*b.z));
}

static bool is_orthogonal(V3 v0, V3 v1, V3 v2)
{
	const double tolerance = 1e-3;
	const V3 n0 = normalize(v0);
	const V3 n1 = normalize(v1);
	const V3 n2 = normalize(v2);
	const double d = dot(n0, n1) + dot(n0, n2) + dot(n1, n2);
	const bool b = fabs(d) <= tolerance;
	return b;
}

int main(int argc, char ** argv)
{
#if 1
	// shear
	V3 v0;
	v0.x = -0.482422;
	v0.y = 0.03;
	v0.z = 0.0;

	V3 v1;
	v1.x = 0.0;
	v1.y = 0.50871;
	v1.z = 0.153075;

	V3 v2;
	v2.x = 0.0;
	v2.y = 2.38419e-07;
	v2.z = 2.24829;
#else
	// orthogonal
	V3 v0;
	v0.x = -0.482422;
	v0.y = -0.0;
	v0.z = 0.0;

	V3 v1;
	v1.x = 0.0;
	v1.y = 0.457492;
	v1.z = 0.153075;

	V3 v2;
	v2.x = -0.0;
	v2.y = -0.752269;
	v2.z = 2.24829;
#endif
	const bool b = is_orthogonal(v0, v1, v2);
	std::cout << "orthogonal = " << (b ? "yes" : "no") << std::endl;
	return 0;
}

Correction for Nifti file: ITK IO preferred qform for the file with shear matrix.

Screenshot at 2022-07-31 09-29-53

For reference, the CT with gantry tilt (that results in a sheared matrix) that @issakomi refers to is available in DICOM and NIfTI format here. You can use dcm2niix to convert it to either NIfTI or NRRD. Since many tools do not handle shear, dcm2niix will also create a copy of the image where the image is interpolated to a orthogonal grid (with the _tilt extension to the file name). This provides an example of what the image should look like.

You can view the image in voxel space (where it appears distorted) or world space (which adjusts for gantry tilt) using the NiiVue live view web page - just drag and drop the NIfTI or NRRD image you wish to inspect. Use the world space button to choose between world (top) and voxel (bottom) space:

@lassoan now that ITK supports the same spatial transforms as other tools, I suspect the odd behavior you have been experiencing will diminish dramatically. In cases where the spatial transform was improperly specified, the user will get similar feedback regarding the odd setting regardless of the tool that is used. I also wonder if we could insert a bit of logic into ITK to detect malformed spatial transforms. With NRRD, the spatial transform is optional, but with NIfTI is is required. Therefore, I assume that reading a NRRD without a spatial transform is loaded with an identity matrix, while the explicit usage of a spatial transform in NIfTI requires the user to supply sensible values. Another thing to check with ITK is whether it can discriminate between NIfTI format files and its ancestor the Analyze format which did not include a spatial transform. Analyze files must be treated in the old way, as described in the NIfTI specification.

My tools tend to do a couple checks to detect a bogus spatial transform (regardless of the format). This logic may be useful for detecting files where the spatial transformation matrix should be used. The basic check is:

  1. All values of the 4x4 spatial transformation matrix must be finite: NaNs, Infinity, negative Infinity suggest a borked matrix.
  2. Assuming a 3D image (at least 2 voxels in each direction): For the 3x3 rotation matrix (e.g. ignoring translations) each row and each column must have at least one non-zero value and the 9 values of the matrix must have some variability.

They (blue) are really nice and match perfectly with the original DICOM slices (red), BTW.

20220731-141731

@issakomi thanks for noticing. dcm2niix does attempt to preserve spatial information. The method is discussed here. It uses in-plane linear interpolation which might not be ideal for all applications. This can mean some loss of high frequencies, but for most uses is less jagged than nearest-neighbor and avoids the ringing artifacts of higher-order interpolation.

1 Like

A note to Slicer users, if you have DICOM images with gantry tilt or variable spacing you can use the Acquisition geometry regularization option is enabled as described here. This creates a nonlinear transform that can later be hardened with whatever options are needed in terms of filters and target sample grid.