Reproduce Segmentation Pipeline from thresholding

Support
1

Hi,

I am currently using 3DSlicer to visualize segmentations coming from another framework.

Here is my pipeline:

  • load a 3D tiff, that contains segmentation by voxels, i.e all voxels with value equals to 10 represents the same segmentation object for example. My segmentations are 3D surfaces.
  • Go to segment editor and create the segmentation using the Threshold button
  • Then I can visualize and export my segmentations in stl format

I’m very happy with the results but would like to apply this process with python (ideally outside 3DSlicer). I’m having a hard time understanding what is the exact process happening inside 3DSlicer.

Could someone help me/ point to me the exact algorithm used for:

  • defining the normals
  • defining the triangles
  • avoid mesh to have holes in it or bubbles

Here is the nice result I get with 3Dslicer:

slicer_result
slicer_result763×768 70.4 KB

And here is the uglier result I get with my current pipeline:

my_result
my_result914×736 133 KB

My current pipeline consists in:

  • creating a point cloud from the point I would use inside the Threshold with 3DSlicer
  • inferring the normal
  • creating the mesh using Poisson algorithm
  • cropping boarders outside initial point cloud
pcd = o3d.geometry.PointCloud()
positions = np.argwhere(original_image==10)
scaling = np.array(original_image.shape)
final_scale = scaling / np.max(scaling)
normalized_position = (2*(positions / scaling) - 1)*final_scale
normalized_position[:,[0,2]] = normalized_position[:,[2,0]]

pcd.points = o3d.utility.Vector3dVector(normalized_position)

distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
radius = 3 * avg_dist

pcd.estimate_normals(search_param = o3d.geometry.KDTreeSearchParamHybrid(radius = radius, max_nn = 10))
poisson_mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_poisson(pcd, depth=10, width=0, scale=1.1, linear_fit=True)[0]

poisson_mesh.compute_triangle_normals()
bbox = pcd.get_axis_aligned_bounding_box()
p_mesh_crop = poisson_mesh.crop(bbox)

What am I missing here? Could someone help me understand what’s the exact pipeline that 3DSlicer is doing?

Thank you very much in advance for your help.

Best

2

If you would like to reimplement the pipeline outside 3D Slicer then have a look at the source code and take whatever you need, it is all open and freely usable. The most relevant parts are here: Slicer/Libs/vtkSegmentationCore at main · Slicer/Slicer · GitHub

We use VTK for all segmentation processing operations, so you may find VTK examples site useful.

1 Like
3

Yes thank you, I know in the end I’ll have to deep dive into the source code.

As I’m new to this I wanted to understand 3DSlicer approach and what makes it much nicer at first sight.

Is there a specific post-processing which smoothes things and make sure that no ‘bubbles’ will appear in the final segmentation? no holes? Is there some sort of outliers removal from point cloud or something like this?

Does it seem a common problem to have artefacts like the ones I have and what makes the result so different than the one from Slicer?

Again, thank you for pointing me the right folder to dig further, but I would really appreciate a general understanding of the differences between my basic pipeline and the one used in 3DSlicer and why?

Really appreciate your support.

4

We represent segmentation as a structured rectilinear grid (volume of voxels), not as a point cloud.

I would recommend to use VTK instead of Open3D. Open3D makes you think that it is some large, mature library developed by Intel for processing any kind of 3D data, but in reality it is a small library primarily developed by a single person in a university research lab (with a few other active contributors), which does a good job in surface reconstruction and alignment. It is not a general 3D processing framework and it is not well positioned to become one.

If you want to implement and maintain your own pipeline outside Slicer then you need to develop a full understanding of all algorithms involved. You can read the source code or use a debugger to step through and inspect everything in Slicer and use VTK library’s excellent resources (examples site, textbook, API documentation) to learn more. I can see that we could assist you in this learning process, but I find it hard to justify this time investment as you are not expected to become a Slicer user or developer. Maybe others can help you out here or in the VTK forum, or you can get paid support from Slicer commercial partners.

5

Sir,

Thank you for your advice and suggestions, they will definitely be useful in the right choice of framework.

Thank you also for taking the time to give me this answer.

As per the justification of time investment, I’ve seen you answering enough questions on this forum to be sure that you know about the joy of learning, teaching and sharing. That you know about the beauty of open source projects and the magnificence of serendipity. I’ve started to pay back my debt to the open source community myself, one brick at a time. Who knows maybe one day I’ll have something to give to 3DSlicer.

But for the moment, I have plenty of readings and work to do to reach that point! Please be sure that your help is much appreciated and that your time is well invested when helping people getting better.

Congratulations for the job accomplished already.

Bests,

Seb

1 Like