Merge CT volume node with a model node in python

I know 3D slicer can merge two volume/slice nodes from separate CT data but is it possible to merge an PLY/STL loaded model node with a CT volume/slice node?

Yes, you can load PLY/STL models and display them with the CT. But because those formats don’t define a reference space they may not align. See this post for example.

Not quite what I was referring to. For example, we obtain 2 nodes as such
ply_node = slicer.util.getNode(‘model_name’)
ct_node = slicer.util.getNode(‘ct_name’)
merged_node = merge(ply_node, ct_node)
such that merged_node contains the image/volume data from both nodes

How do you envision combining models and volumes? Would you like to end up with a model or a volume?

You can import a model into a segmentation and then use Segment Editor’s Mask volume effect (provided by SegmentEditorExtraEffects extension) to paste the segment into the volume.

hey, i found have this from the nightly script repository

meshVolumeNode = getNode(mesh_model)
seg = slicer.mrmlScene.AddNewNodeByClass(‘vtkMRMLSegmentationNode’)
slicer.modules.segmentations.logic().ImportModelToSegmentationNode(meshVolumeNode, seg)

once i have a segment volume node, how can i merge it with a CT volume node?
also what is the function SetAndObserveImageData used for? the VTK documentation doesn’t explain much.

I’m working on an extension that builds around panoramic reconstruction which essentially straightens a volume: Slice rotation in viewport when using SetSliceToRASByNTP in python
but i also want this reconstruction to incorporate mesh models so im trying to merge volumes - im not sure if this is the correct approach to this.


OK, this explains everything.

You can create 3D panoramic reconstructions using this recently added module: How to implement CPR (curved plannar reconstruction) from centerline?

To display models in the same way, you can import the model to segmentation node, from there export as a labelmap volume node, and then apply curved planar reconstruction to the labelmap volume.

An alternative approach would be to create a grid transform from the transformed corner points of the slices, but this would require a little bit of Python scripting. This method would be particularly useful for transforming sparse point sets (landmark points, lines, curves), which cannot be well represented by image volumes. It would be nice if you could work on this we can help you to get started.

Woa this is really neat! I didn’t realize this feature had been implemented. Ultimately I’m trying to do implant planning so having the an stl/ply mesh representation of the implant shown in the pan tomographic representation would be really useful.
Can I merge segmentation with ct node in order to use it as an input volume for CPR?
Or can multiple volume node inputs be implemented?

You can “paint” an STL model into a CT by importing the STL model into a segmentation then use Mask volume effect (provided by SegmentEditorExtraEffects extension) in Segment Editor module.

Model painted into CBCT volume using Mask volume effect:

The Mask Volume works beautifully with the CPR extension. One caveat I have though is that when I use mask volume, the segment color turns gray once projected onto the CT volume. so when I use the CPR extension to transform the gray color of the segment blends against the gray values of the CT volume.
any workaround for that?

In mask volume effect, you can pick the gray color intensity. Probably you want to set a high value to indicate that the implant is very dense material.

i saw on the nightly script repository an example on how to access scripted modules. I want to access the mask volume effect, i am trying this
but i get that module ‘modules’ has no attribute ‘segmenteditoreffect’

You can learn from these examples how to use Segment Editor effects from scripts:

1 Like