Multi-Class Segmentation with CT + Aligned STL (Dental Dataset)

[abhijeet2410 (Abhijeet Thakare)]

Operating system: Window
Slicer version: 3 D Slicer 5. 10.0

---

Question: Multi-Class Segmentation with CT + Aligned STL (Dental Dataset)

I am working on a dental AI dataset pipeline where I have:

• CT scan (DICOM volume)

• Aligned STL models (upper jaw, lower jaw, bite scans)

I am able to visualize everything correctly using VTK:

import vtk

def load_stl(path, color):
    reader = vtk.vtkSTLReader()
    reader.SetFileName(path)
    reader.Update()

    mapper = vtk.vtkPolyDataMapper()
    mapper.SetInputConnection(reader.GetOutputPort())

    actor = vtk.vtkActor()
    actor.SetMapper(mapper)
    actor.GetProperty().SetColor(color)

    return actor

renderer = vtk.vtkRenderer()

upper = load_stl("UpperJawScan.stl", (1,1,1))
lower = load_stl("LowerJawScan.stl", (0.9,0.9,0.9))

renderer.AddActor(upper)
renderer.AddActor(lower)

So currently:

• CT volume and STL teeth are spatially aligned

• STL provides clear tooth geometry

• CT provides voxel grid (needed for training)

---

Goal : I want to create a multi-class segmentation dataset such as:

Class ID	Structure
0	Background
1	Teeth
2	Gum
3	Impacted Teeth

---

Problem

• Teeth are not clearly visible in CT slices, but are very clear in STL

• STL is surface mesh, not voxel labels

• Exporting via vtkOBJExporter only gives geometry, not labels

---

Core Question

How can I generate a voxel-wise multi-class segmentation mask from aligned STL + CT data using tools like 3D Slicer or ITK-SNAP?

This is aligned STL + CT data. How to annotate it using 3D slicer ? White part show STL and transparent part show the CT

---

Constraints

• Dataset is for deep learning (3D segmentation / PointNet / UNet)

• Prefer semi-automatic or automated pipeline

---

What I’ve Tried

• Rendering CT + STL together → works

• Exporting meshes → no label info

• Manual annotation → difficult because CT does not clearly show teeth