Visualizing 3D bounding boxes in Slicer Markups (Fiducials vs Lines) + JSON format question

Support
1

Description:

I am working with 3D Slicer markups to debug an interactive model that uses prompt-based inputs (3D bounding boxes). My goal is to ensure that the voxel → world coordinate transformation is correct and that the prompts being sent to the model correspond exactly to what is visualized in Slicer.

Currently, I generate a markup JSON file from bounding boxes by converting each box into its 8 corner points:

def bbox_to_points(bbox):
    (x0, x1), (y0, y1), (z0, z1) = bbox

    return [
        (x0, y0, z0),
        (x0, y0, z1),
        (x0, y1, z0),
        (x0, y1, z1),
        (x1, y0, z0),
        (x1, y0, z1),
        (x1, y1, z0),
        (x1, y1, z1),
    ]

Then I serialize them into a Slicer Markups JSON file:

def save_prompts_as_json(prompts_dict, affine, out_path):

    control_points = []

    for label, bboxes in (prompts_dict.get("pos_bboxes") or {}).items():
        for idx, bbox in enumerate(bboxes):
            for pt in bbox_to_points(bbox):
                control_points.append({
                    "label": f"BOX-POS-label={label}-idx={idx}",
                    "position": [float(coord) for coord in voxel_to_world(pt, affine)]
                })

    output = {
        "@schema": SCHEMA,
        "markups": [
            {
                "type": "Fiducial",
                "coordinateSystem": "RAS",
                "controlPoints": control_points
            }
        ]
    }

    with open(Path(out_path) / "prompts.json", "w") as f:
        json.dump(output, f, indent=2)

This works in the sense that all points are correctly displayed in Slicer, and I can verify the coordinate transformations.

However, the bounding box is only visible as 8 independent fiducial points (see the image below, with a sample from Totalsegmentator_dataset_v201). This is sufficient for validation, but not ideal for clear visualization or presentations.

image
image1920×1080 198 KB

What I would like is either:

  • A way to represent a true 3D bounding box directly in Slicer Markups, or
  • A way to connect these points with edges (wireframe box) inside Slicer

If possible, it would also be very helpful to provide an example of how to implement the line connections between the 8 points (i.e., defining the 12 edges of the bounding box) using Slicer Markups JSON or Python API.

Question:
What is the recommended way to visualize 3D bounding boxes in Slicer Markups for presentation purposes?

  • Is there a supported way to represent a wireframe box directly in Markups JSON?
  • Or should I construct edges manually using additional markup types?
  • If so, what would be the cleanest way to extend my current JSON generation approach?

Any guidance or example Python code or JSON would be very helpful.

2

Yes, Slicer has a native Markups object for bounding boxes, they are vtkMRMLMarkupsROINode objects. They are specified by a center, axis directions, and box dimensions rather than by corner point locations, but it is straightforward to convert between these representations. Here is an example of the contents of a saved ROI in .mrk.json format:

{
    "@schema": "https://raw.githubusercontent.com/slicer/slicer/master/Modules/Loadable/Markups/Resources/Schema/markups-schema-v1.0.3.json#",
    "markups": [
        {
            "type": "ROI",
            "coordinateSystem": "LPS",
            "coordinateUnits": "mm",
            "locked": false,
            "fixedNumberOfControlPoints": false,
            "labelFormat": "%N-%d",
            "lastUsedControlPointNumber": 1,
            "roiType": "Box",
            "center": [-11.136674880981445, -88.3449478149414, 1371.537353515625],
            "orientation": [-1.0, -0.0, -0.0, -0.0, -1.0, -0.0, 0.0, 0.0, 1.0],
            "size": [41.72665258450965, 43.168723933624904, 91.27469589070517],
            "insideOut": false,
            "controlPoints": [
                {
                    "id": "1",
                    "label": "Crop Volume ROI-1",
                    "description": "",
                    "associatedNodeID": "",
                    "position": [-11.136674880981445, -88.3449478149414, 1371.537353515625],
                    "orientation": [-1.0, -0.0, -0.0, -0.0, -1.0, -0.0, 0.0, 0.0, 1.0],
                    "selected": true,
                    "locked": false,
                    "visibility": true,
                    "positionStatus": "defined"
                }
            ],
            "measurements": [
                {
                    "name": "volume",
                    "enabled": false,
                    "units": "cm3",
                    "printFormat": "%-#4.4g%s"
                }
            ],
            "display": {
                "visibility": false,
                "opacity": 1.0,
                "color": [0.4, 1.0, 1.0],
                "selectedColor": [1.0, 0.5, 0.5],
                "activeColor": [0.4, 1.0, 0.0],
                "propertiesLabelVisibility": true,
                "pointLabelsVisibility": false,
                "textScale": 3.0,
                "glyphType": "Sphere3D",
                "glyphScale": 3.0,
                "glyphSize": 5.0,
                "useGlyphScale": true,
                "sliceProjection": false,
                "sliceProjectionUseFiducialColor": true,
                "sliceProjectionOutlinedBehindSlicePlane": false,
                "sliceProjectionColor": [1.0, 1.0, 1.0],
                "sliceProjectionOpacity": 0.6,
                "lineThickness": 0.2,
                "lineColorFadingStart": 1.0,
                "lineColorFadingEnd": 10.0,
                "lineColorFadingSaturation": 1.0,
                "lineColorFadingHueOffset": 0.0,
                "handlesInteractive": true,
                "translationHandleVisibility": true,
                "rotationHandleVisibility": false,
                "scaleHandleVisibility": true,
                "interactionHandleScale": 3.0,
                "snapMode": "toVisibleSurface"
            }
        }
    ]
}