Misalignment between Eigen Artemis DICOM image and segmentation

[sulaimanvesal]

Hi,

I wanted to ask this point, that why my segmentation output during export placed at the corner of NRRD volume. Even though the volume origin is at (0.0, 0.0, 0.0).
Below is the code that I am using to export a vtk model to a segmentation label map.

# Center the view
layoutManager = slicer.app.layoutManager()
threeDWidget = layoutManager.threeDWidget(0)
threeDView = threeDWidget.threeDView()
threeDView.resetFocalPoint()

usPath =  "/Volumes/REC00000" 
segPath = "/Volumes/SURFRCON.vtk"

volumeNode   = slicer.util.loadVolume(usPath)
prostateNode = slicer.util.loadModel(segPath)

seg = slicer.mrmlScene.AddNewNodeByClass('vtkMRMLSegmentationNode')
seg.SetReferenceImageGeometryParameterFromVolumeNode(volumeNode)
slicer.modules.segmentations.logic().ImportModelToSegmentationNode(prostateNode, seg)
# seg.CreateBinaryLabelmapRepresentation()

labelmapVolumeNode1 = slicer.mrmlScene.AddNewNodeByClass('vtkMRMLLabelMapVolumeNode')
slicer.modules.segmentations.logic().ExportVisibleSegmentsToLabelmapNode(seg, labelmapVolumeNode1, volumeNode)

outputpath = "/Volumes/test.nrrd"
slicer.util.saveNode(labelmapVolumeNode1, outputpath)
slicer.mrmlScene.RemoveNode(labelmapVolumeNode1)
slicer.mrmlScene.Clear(0)

This is the output of GetOrigin(), GetSpacing() for reference volume (volumeNode) and exported NRRD (test.nrrd) .

test.nrrd:(0.0, 0.0, 0.0) (0.37800000000000006, 0.37800000000000006, 1.0)
REC00000: (0.0, 0.0, 0.0) (0.37800000000000006, 0.37800000000000006, 1.0)

This is also the nrrd output. As you can see the segmentation map is centered in upper right corner.

[sulaimanvesal]

Hi @lassoan

Could you please help me with this problem?

[lassoan (Andras Lasso)]

Volume origin is defined in physical space: IJK=(0,0,0) voxel content will be whatever is in the segmentation at the physical RAS=(0,0,0) position.

[sulaimanvesal]

Thank you @lassoan , I have checked my data and as you correctly mentioned, my volumeNode is in IJK coordinate and my vtk mesh file is in RAS coordinate (with different origin and pixel spacing).

I looked in the forum on how to convert RAS to IJK coordinates but could not really make it. I was wondering if there is a python example that can help me!

Another question is that should I export my vtk mesh file from RAS to IJK before ExportVisibleSegmentsToLabelmapNode or after?

[lassoan (Andras Lasso)]

You can find example to IJK<->RAS conversions in the script repository.

[sulaimanvesal]

Thanks @lassoan.

I could figure it out the problem and could align the volume with segmentation label map using the following piece of code (I know centering the volume in this way is not recommended but since I am just exporting the label maps to NRRD for my segmentation task, and it solved my issue):

# Center volume 
if center_volume:
     volumesLogic = slicer.modules.volumes.logic()
     volumesLogic.CenterVolume(volumeNode)

I explored my multi-frames DICOM images and I noticed that I am working with raw data. It means the DICOM header does not have the PixelSpacing tag, but that information stored in metadata as a private tag. When I load the images into the 3D Slicer, it automatically map the volume to a pixel spacing of [1.0 , 1.0, 1.0] mm and origin of [0.0, 0.0, 0.0].

I was wondering if there is any way, that we can tell the Slicer to pick the Pixel-Spacing from the private tag when we load the data? I tried to resample the data but it was not a practical approach.

1. I realized, what if I add a new tag to DICOM headers of my RAW DICOMS as “PixelSpacing” and copy the information from private tag and store it as a new DICOM file, would it load on 3D Slicer correctly? I am not sure.

   (1129, 1004) Private tag data IS: [67, 3899, 2884, -4096]
   (1129, 1005) Private tag data IS: [67, 3899, 543, -4096]
   (1129, 1008) Private tag data DS: “0.0”
   (1129, 1015) Private tag data IS: “4096”
   (1129, 1016) Private tag data DS: [0.24960000813007, 0.24960000813007, 0.24960000813007]
   (1129, 1017) Private tag data DS: “4.0”
   (1129, 1023) Private tag data IS: “4”

Best.

[lassoan (Andras Lasso)]

In general, basic image metadata, such as pixel spacing must not be stored in private tags. Unfortunately, some imaging vendors do use private tags (and it takes time to convince them to better adhere to standards) and for these cases we add DICOM plugins to retrieve and use metadata in private fields.

The (1129, 1016) tag is used by Eigen Artemis 3D ultrasound systems to store pixel spacing information. We already have a plugin for this device in SlicerHeart extension. All you need to do is to install SlicerHeart extension and the DICOMUltrasoundPlugin will recognize this image and load it correctly.

[sulaimanvesal]

Thank you @lassoan for your always support. You are correct, my images come for Eigen Artemis 3D ultrasound system.
I have installed SlicerHeart as you said and I can see it in the list of plugins on DICOM module when I load my DICOM images. However, it does not work and it loads the data as 2D image only rather multi-frame 3D like before. When you go to volume module the origin and spacing again pointing at [1.0, 1.0, 1.0] and [0.0 ,0.0, 0.0].

I copied the python function examineEigenArtemis3DUS form DICOMUltrasoundPlugin for troubleshooting and tested manually on Jupyter. For outputSpace and OutputOrigin, it gives me the correct conversion when I give a test sample:

OutputSpace: [0.24960000813007, 0.24960000813007, 0.24960000813007] 
OutputOrigin: [61.90080201625736, 61.90080201625736, -36.31680118292518] 

I was wondering, if I can use this new OutputSpace and OutputOrigin to enforce Slicer python use them as the coordinates when I call slicer.util.loadVolume, rather than the default spacing and origin.

[lassoan (Andras Lasso)]

Make sure your SlicerHeart extension is up-to-date. Either install latest Slicer Stable Release and update SlicerHeart extension; or install latest Slicer Preview Release and install SlicerHeart extension.

For example, Artemis images that are available in TCIA load perfectly with latest Slicer Preview Release:

[sulaimanvesal]

Hi @lassoan

I downgraded my Slicer to up-to-date stable version. To make sure everything works perfectly, I downloaded the TICA data as well.

Here is as you see: the top row is the same image that you loaded and the second row is our internal data (both Artemis).

One thing I noticed that when I load our own data, gets the following warning:

This is the metadata tag list that we have for my internal DICOM (sorry I can’t send you a sample), I thought maybe there is a difference in terms of tags naming or something else that give the warning and cause this problem in DICOMUltrasoundPlugin.

Dataset.file_meta -------------------------------
(0002, 0002) Media Storage SOP Class UID         UI: Ultrasound Multi-frame Image Storage
(0002, 0013) Implementation Version Name         SH: 'EIGEN_361'
(0002, 0016) Source Application Entity Title     AE: 'ARTEMIS'
-------------------------------------------------
(0008, 0020) Date                          DA: '2015'
(0008, 0064) Conversion Type                     CS: 'SYN'
(0008, 0070) Manufacturer                        LO: 'Eigen'
(0018, 1020) Software Versions                   LO: '2.3.0.81'
(0018, 106a) Synchronization Trigger             CS: 'EXTERNAL'
(0018, 5010) Transducer Data                     LO: ['Hitachi', 'Noblus', 'Hitachi C41V']
(0020, 0200) Synchronization Frame of Reference  UI: Universal Coordinated Time
(0028, 0002) Samples per Pixel                   US: 1
(0028, 0004) Photometric Interpretation          CS: 'MONOCHROME2'
(0028, 0008) Number of Frames                    IS: "291"
(0028, 0009) Frame Increment Pointer             AT: (0054, 0080)
(0028, 0010) Rows                                US: 496
(0028, 0011) Columns                             US: 496
(0028, 0034) Pixel Aspect Ratio                  IS: [249, 249]
(0028, 0100) Bits Allocated                      US: 8
(0028, 0101) Bits Stored                         US: 8
(0028, 0102) High Bit                            US: 7
(0028, 0103) Pixel Representation                US: 0
(0028, 1050) Window Center                       DS: "50.0"
(0028, 1051) Window Width                        DS: "0.0"
(0054, 0081) Number of Slices                    US: 291
(1129, 0010) Private Creator                     LO: 'Eigen, Inc'
(1129, 1002) Private tag data                    LO: 'EigProc67'
(1129, 1008) Private tag data                    DS: "0.0"
(1129, 1016) Private tag data                    DS: [0.24960000813007, 0.24960000813007, 0.24960000813007]
(1129, 1017) Private tag data                    DS: "4.0"

[lassoan (Andras Lasso)]

You can fix up the current plugin to make it read your images as well. It is just a Python script and you can attach a debugger to go through the code step by step to see why it is not recognized.

If that does not work for you then please acquire image of a phantom (or even an empty water tank) and send us that image.

[sulaimanvesal]

Thanks a lot @lassoan

I found the error after debugging the python code. In my Ultrasound DICOM images the PixelSpacingPrivateTag store the values as a list e.g. [0.17, 017, 017], while in TCIA data the PixelSpacingPrivateTag value is a single value e.g. 0.17. The code needs to be updated so it can consider both list and single value otherwise float () throw the error.

This was the fix that I made in the current plugin, not a very fancy way as the return of ds[pixelSpacingPrivateTag].value is sometimes object:

try:
   pixelSpacingPrivate = float(ds[pixelSpacingPrivateTag].value)
except:
   pixelSpacingPrivate = ds[pixelSpacingPrivateTag].value
   pixelSpacingPrivate = float(pixelSpacingPrivate[0])

I am almost there, this fix nicely solved my issue with TCIA data and STL files alignment (the one that I had problem with centering the volumes). I can’t understand why the vendors don’t follow the same standards for DICOM header, that could save alot of time.

Just one last small problem still here. My local data has gland segmentation as vtk file. If you look at the screenshot it looks like the axial/coronal/sagittal views for vtk model miss-placed. The vtk file generated originally based on the same Ultrasound volumes. Do you have any suggestion on how to correct this issue. I checked the RAS and LPS version when I load vtks but none of them correct the issue.

As I said, this problem is not exist anymore with TCIA data and they are nicely aligned.

[lassoan (Andras Lasso)]

If you have 3 pixel spacing values then read all 3 and set them in the volume node as x, y, z spacing. If it works well then please send a pull request. Please also add a comment that Artemis Eigen 2.3.0.81 software version generates such images.

[sulaimanvesal]

Actually, the pixel spacing values in my data are always the same. However, I set them in the volume node as x,y,z, but it gives a warning and Slicer load them with default origin and spacing. There should be a fix for this probably.

Regarding the issue that i mentioned for my local data miss-alignment, the following transformation with rotating the model [-90, 0.0, -90], aligned the vtk with Ultrasound volume. However, I wish Slicer could solve this rather than applying transformation.

I will create a pull request on Github though and report the fix for Artemis.

[lassoan (Andras Lasso)]

@sulaimanvesal

However, I set them in the volume node as x,y,z, but it gives a warning

Can you copy here the entire warning message?

@sulaimanvesal

Regarding the issue that i mentioned for my local data miss-alignment, the following transformation with rotating the model [-90, 0.0, -90], aligned the vtk with Ultrasound volume.

In medical image computing, LPS coordinate system is used most commonly (x=patient left, y=patient posterior, z=patient superior). Therefore if directions of an image or model are known relative to a patient then it makes the most sense to use this convention.

In ultrasound imaging, the probe can be placed on the patient in various orientations, so often you have no clue how image axes relate to patient axes. However, due to human anatomy, endocavity ultrasound probes are quite constrained in their insertion direction, so you can have a good approximation for image to patient matrix for a specific device. This is why the Artemis DICOM plugin applies a rotation in its IJK to RAS matrix:

SlicerHeart/SlicerHeart/blob/d8f9b6e45ddc3b869046c151210f3dae0b80eb82/DicomUltrasoundPlugin/DicomUltrasoundPlugin.py#L800-L810

I understand that this rotation may be unexpected, but it is not arbitrary and I would encourage you to apply the same rotation matrix to the model loaded model.

The only better solution I can think of is would be if Eigen better adhered to DICOM standard and specified image axes in LPS coordinate systems in the first place.

@fedorov Are you in contact with Eigen? Do you know about this ambiguity of the spacing field content (sometimes contains one value, sometimes contains a sequence of 3 values)? Do you know if they are aware that their lack of following DICOM standard or conventions is causing trouble for users?

[fedorov (Andrey Fedorov)]

@lassoan

@fedorov Are you in contact with Eigen? Do you know about this ambiguity of the spacing field content (sometimes contains one value, sometimes contains a sequence of 3 values)? Do you know if they are aware that their lack of following DICOM standard or conventions is causing trouble for users?

@lassoan no, I am not in constant contact with Eigen. I do know someone at Eigen, who may have more insight on this, and I will forward the thread. However, based on my experience, I don’t recall when vendors were eager to address this kind of issues raised by researchers. For anything to change, those should be raised by paying customers. I am afraid the reality is that 1) lack of following DICOM standard or conventions as in this example often does not cause troubles for paying users (clinical users); 2) attempts to follow the standard precisely may cause troubles for paying users, because the downstream systems working with the data may not be able to deal with the proper implementation of the standard.

I agree we all should keep raising those issues with the vendors. But most importantly, I hope that paying customers do this more often.

[sulaimanvesal]

Hi @lassoan,

Sorry for not responding as I was kinda sick for past few days. Thank you so much for detailed response.

Actually, I have solved all my issues. I found the reason behind that missalignment between the 3D Ultrasound image and mesh file after correcting the pixel-spacing and origin. For Eigen Artmeis ultrasound images, there is no tag for patient orientation information and they don’t even store it as private tag. So I needed a transformation ([0, 1, 0, 0, 0, 1, 1, 0, 0] to apply to all mesh files, and it worked well (I saw now your comment that you also pointed this out).

One more bug that I found in “DICOMULtrasoundPlugin” for Eigen Artemis function that it always takes the private tag as final output-spacing. However, in my dataset I had images that had both standard ds.PixelSpacing tag and also private tag (the values were different, and the correct spacing was the one from ds.PixelSpacing). So, for these type of the images the code should choose the default ds.PixelSpacing to correctly show the images. I have corrected in my version of plugin code, and I thought to share it here also. I am not sure, how common this can be, but for my dataset I found some cases.