We have been successfully using SAM to create partial volumes from CT segmentations, track these volumes, and apply the resulting transformations to AUT models in MATLAB to visualize motion.
We are now trying to repeat the same workflow using STL implant models instead of CT segmentations. Our workflow in 3D Slicer (version 5.10.0) is:
Load the implant STL model into 3D Slicer.
Convert the model to a segmentation node.
Export the segmentation node as a binary labelmap volume.
In the SAM pre-processing module, set the Volume Node to the binary labelmap and the Segmentation Node to the STL-derived segmentation.
The AUT models, volumes, and transforms are exported successfully. However, after tracking the generated TIFF volumes, applying the resulting transformation matrices to the AUT models produces an incorrect implant orientation.
While investigating, we noticed that the exported TIFF volumes appear to have a different orientation from the original STL model (specifically, a 180° rotation about the Y-axis). In contrast, the TIFF volumes generated from our CT segmentations retain the orientation of the source CT data. The images below show the STL (left) and CT (right) derived models and their associated TIFF volumes from the pre-processing module. Note: the TIFF volumes have been scaled back to match the same dimensions as the models.
Could this orientation mismatch be responsible for the incorrect tracking results? If so, is there a recommended workflow for using STL models with SAM that preserves the correct orientation, or an alternative approach that would avoid this issue?
Thanks for reaching out. I do think that the orientation mismatch may be responsible for the incorrect tracking results. We have run into this before, where there was a difference in orientation between .stl bone models and the exported tiffs. In our case the tiff volume was flipped along the y and z axes relative to the .stl models. When generating the configuration file you can set a “Volume Flip,” and this can also be manually changed in the .cfg file. I would give this a try and retrack the file. Let me know if this resolves the issue.
I failed to mention this in my previous post, but I had initially tried a [1 0 1] flip since the TIFF seemed to be flipped about the Y-axis, but I had no success. I tried your suggestion of a [0 1 1] flip but the tracking is still off.
After further investigation, I managed to create a transform in Slicer that flips the TIFF to match the orientation of the STL. My initial statement that the TIFF is flipped about the Y-axis was incorrect, and the TIFF model is actually flipped about the Z-axis, requiring a [1 1 0] flip.
I retracked the first frame with this flip, and it seems the AUT models are in the correct orientation now, but still not properly translated. I have tried to apply some of the transforms the partial volume generator provides, but I have had no success there either.
I wanted to provide some more information for you to help us pinpoint the issue.
In Slicer, I uploaded the AUT model and the associated TIFF volume. After applying the PVOL2AUT transform, the TIFF and AUT model do not align (I can confirm this transform works with CT model datasets).
The image of the TIFF in AUT space is below, and you can see that it is still a reflection of the AUT model. Applying a 180-degree rotation about the Z-axis (which I believe would be a [1 1 0] flip in the cfg file) would get the TIFF in the correct orientation. However, I would still need to translate the TIFF along the Y-axis by some arbitrary value of ~-85 mm? Not sure where this 85 would come from.
I tried to apply this transform to the AUT models in MATLAB, and then apply the tracking, but still no success. Perhaps I am thinking of these transformations incorrectly.
I appreciate the help thus far.
UPDATE: After posting this, I quickly swapped the -85 mm translation to be positive. The models seem to align perfectly now. However, I am still not sure where this 85 mm originates from. I am trying to check the volume dimensions, transforms, etc. but this value is nowhere to be found.
We have not used SAM to process and track implants before, so my experience here is limited. I haven’t run into this issue before with our human data. We did just collect some implant data, so I will try to track it and see if we have a similar issue.
Sounds good, thanks. I did provide an update in my last post in which I solved the issue. However, my solution required me to manually translate the model 85 mm until it matched closely. So, to acquire a more accurate solution is still required. I will keep looking into this and provide the solution if I find it.
Good news: I have found the solution with my STL models in case you (or anyone else) run into the same issue. Briefly, the PVOL2AUTO transform that was exported did not compensate for the additional Z-axis rotation that needed to occur.
The original PVOL2AUTO transform provided a 180-degree rotation about the X-axis and an additional translation about the Y-axis of -42.79 mm:
PVOL2AUT =
1 0 0 0
0 -1 0 -42.79
0 0 -1 0
0 0 0 1
However, this transformation failed to account for the additional Z-axis rotation and translation about the Y-axis of about -85 mm (which I realized is actually just double the original Y-axis translation in PVOL2AUT):
Tz =
-1 0 0 0
0 -1 0 -42.79*2
0 0 1 0
0 0 0 1
By combining the two transforms, you essentially form a rotation matrix representing a 180-degree rotation about the Y-axis with a Y-axis translation of the original magnitude:
Ty =
-1 0 0 0
0 1 0 -42.79
0 0 -1 0
0 0 0 1
Therefore, in MATLAB I simply applied Tz to the AUT models to ensure that they are in the same space as the TIFF when imported into SAM. The tracking is then bringing the new models into the correct position.
Glad you found a solution. This jogged my memory a little bit. This was a while ago, but I remember now that we encountered problems with the transforms and applying the tracking to models when developing our pipeline in MATLAB. This stemmed from MATLAB and Slicer not using the same coordinate systems, which had to be accounted for. 3DSlicer uses Right-Anterior-Superior (RAS), whereas MATLAB uses Left-Posterior-Superior (LPS). This is documented here.
Yes, I did forget to mention that I needed to convert that Tz matrix to IJK in Matlab! That’s why the translation becomes positive rather than negative.
I do wonder why the STL models are processed differently than the CT based models. Did you ever get a chance to try out your recent implant data?
I did and ran into the same problems when trying to use our pipeline for CT based models. The models required additional translations in the y- and z-axes, but did line up. I’m not sure why these models would be processed differently, but is something we will look into.
