Hi, anybody know how to take the segmentation of the 3DSlicer scene and put those same segmentation on Unity3D with the same characteristics (size, dimension, position of objects), please? I need this for a Project in Zspace screen which works with Unity3D
Slicer can export the segmentation to obj format (and some others) that Unity can use.
The length unit is mm by default. You can change the scale in the export window but I would recommend keeping the default, as mm is used as length unit by DICOM and all clinical imaging software.
Note that while Unity is an awesome gaming engine, it is quite poor for medical imaging applications. For example, you may find that Unity cannot import the full-quality meshes with all details (it has a limit on maximum polygon count per mesh). If you run into this limitation but still want to work with Unity then you either need to split the mesh to multiple pieces or decimate it before exporting it to file in Slicer (export segments to model, decimate using Surface Toolbox module, then save the models using File / Save data).
1 Like
So let’s imagine I have a segmentation scene file and I want to put these segmentations in unity3D from slicer without changing the characteristics of the segmentations. So how do I do this? because concretely I think it is possible but I cannot find exactly what to do.
You can use use SlicerOpenAnatomy extension’s OpenAnatomy export module, which can export a segmentation as a set of low-poly models by a single click of a button in OBJ or glTF format.
Once you have the exported OBJ files, if you drop them into the Assets folder (or a subdirectory of the Assets folder if you want to keep things more organized) of your Unity project, Unity will automatically import them as assets. Then you can add them to your scene within Unity. It’s been a while since I’ve done it, but I recall the import process working smoothly. You need both the .obj file and the associated .mtl file to bring along the associated color you have from Slicer. The obj file holds the mesh and the mtl file holds the material properties, which includes color.
1 Like
About the scaling, the Slicer export has units of mm, and I think the Unity default might interpret numbers as meters, but scaling things in Unity is very easy, and if your objects are 1000x too big, just scale them down. For what I was doing, only relative sizes mattered and all meshes were coming from Slicer, so I didn’t end up having to change anything.
1 Like
If you want to avoid changing the segmentation to a mesh model and display the segmentation as a volume in unity - there are also several options to do this.
Basically speaking - you can export the segmentation as a sequence of 2D slices and import them into a 3D texture. The same way as volumetric clouds are rendered.
If you want to use DICOM you might need to write an import plugin that will read and convert the files to a texture sequence or a 3D texture, but if you just want simple visualisation you can export a sequence of images using the Screen Capture module and use the sequence with volume renderers like this one.
There were a couple of very nice looking prototypes to do volume rendering of medical images in UE4 that appeared to show better performance - so Unity is not the only game engine that can give you this functionality.
Edit: there is a Simple DICOM Loader in the Unity Asset Store.
Thank you everyone, you have helped me a lot.
I have another question. How do I PRECISELY transfer the volume rendering of the sections I have on 3DSlicer to Unity3D without changing the orientation, the scale etc … PLEASE ??
yes I have a complete Proyecto fist you need to create a 3D model and save it as .ply file and go to this website so you don’t need Blender to convert to fix the file
this is because is the format that unity needs.
https://products.aspose.app/3d/conversion/ply-to-fbx
after converting your 3d model format you can drag it in unity I guess you want to do an AR or something like it, if that’s the case feel free to ask me
thanks , Basically I have MRI sections that I transferred from 3Dslicer to unity3D and now I want to have the volume rendering of these images in unity3D, how do I do that?
they are in .PNG format
pleas open Slicer and see the name of your file you can see these by going to Data in Modules
after that pleas change the name of your file mi file name is aseg.presurf-IXI061-A
so try this:
You can open python code by crl+3
nodo='aseg.presurf-IXI061-A'
sliceLogic = slicer.app.layoutManager().sliceWidget('Red').sliceLogic()
compositeNode = sliceLogic.GetSliceCompositeNode()
compositeNode.SetLinkedControl(1)
#link the slice control
slicer.util.getNode('vtkMRMLSliceNodeRed').SetUseLabelOutline(1);
#Thickness line
labelMapNode=getNode(nodo)
labelMapNode.GetDisplayNode().SetSliceIntersectionThickness(1)
#####3d
seg=slicer.mrmlScene.AddNewNodeByClass('vtkMRMLSegmentationNode')
slicer.modules.segmentations.logic().ImportLabelmapToSegmentationNode(getNode(nodo), seg)
seg.CreateClosedSurfaceRepresentation()
After this code you will have a 3d model
You will have to go to segment edit click segmentation
after click go to this menu
and you will have you 3d protect in stl file
Implementing medical image viewer in Unity is a huge pain. You essentially need to redevelop everything from scratch. You find a few very basic volume rendering packages that you should be able to set up a simple demo with it, but they are nowhere near to what real users would expect as a minimum requirement for a clinical software.
I would only recommend Unity for very special use cases, for example for building simulation-based training applications, where the emphasis is on fluid computer-game-like immersive experience and you don’t need clinical-grade visualization and tools.
1 Like
I have this error
>>> #####3d
>>> seg=slicer.mrmlScene.AddNewNodeByClass('vtkMRMLSegmentationNode')
>>>slicer.modules.segmentations.logic().ImportLabelmapToSegmentationNode(getNode(nodo), seg)
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: arguments do not match any overloaded methods
>>> seg.CreateClosedSurfaceRepresentation()
True
First argument of ImportLabelmapToSegmentationNode must be a labelmap volume. See more examples here.
Yes I know but I have to do this on unity3D even if the result is not perfect but I have to show something to my professor
Actually I already have my images in unity3D so now from unity3D I have to do the volume rendering, thanks to a plugin I guess
Or how use the surface rendering instead of volume rendering in unity3D ?
You would load the image into Slicer, use Segment Editor to segment all the structures that you are interested in, and export the segmentation to OBJ files (see step-by-step instructions here). You can import the OBJ files into Unity.