Controlling Volume Reslice Driver to follow 5DOF inplane view of tool

Support
, , ,
1

I’d like show a 5DOF inplane view of needle using the Volume Reslice Driver extension, with the 6th DOF rotation locked in alignment with respect to a canonical axis in the global RAS reference frame, providing a more grounded view of tool paths that remain orientated with respect to the volume.

While the VolumeResliceDriver easily allows for controlling slice views relative to 6DOF pose transforms, as well as with respect to 3 additional orthogonal axis - either fixed to the global RAS reference frame or the relative to tracked transform itself, the fixed RAS modes are useful in only tracking the 3DOF point of a tool tip, while the relative inplane modes are a little too disorientating for real time use. Thus, I’d like to replicate a hybrid of the two modes, that may be common among many IGT visualizations, by locking one rotation axis of an inline slice view to a chosen RAS axis.

image
image1920×1080 149 KB

image
image1920×1080 153 KB

Via scripting, I could publish the aligned transforms for each RAS axis variant for a given tool:

"""
============================
Position sending server
============================

Simple application that starts a server that provides a stream of tool trajectories.

from slicer.util import pip_install
pip_install("pyigtl")

Adapted from:
https://github.com/lassoan/pyigtl/blob/d28132d8e9e1cef4ac531d795f5284c8aa739408/examples/example_position_server.py
https://github.com/lassoan/pyigtl/blob/d28132d8e9e1cef4ac531d795f5284c8aa739408/pyigtl/tests/test_basic_comm.py#L87-L93
"""

import pyigtl 

from time import sleep
import numpy as np
from scipy.spatial.transform import Rotation as R

server = pyigtl.OpenIGTLinkServer(port=18944)

timestep = 0
while True:
    if not server.is_connected():
        # Wait for client to connect
        sleep(0.1)
        continue

    # Generate periodic trajectory: Lissajous curve on sphere
    timestep += 1
    radius = 50.0
    center = np.array([0.0, 0.0, -100.0])
    # Use two periodic angles for smooth looping
    phi = (timestep * 0.001) % (2 * np.pi)  # azimuthal
    theta = (timestep * 0.002) % (2 * np.pi) # polar
    x = center[0] + radius * np.sin(theta) * np.cos(phi)
    y = center[1] + radius * np.sin(theta) * np.sin(phi)
    z = center[2] + radius * np.cos(theta)
    position = np.array([x, y, z])

    # Orientation: always point toward center
    forward = center - position
    forward = forward / np.linalg.norm(forward)
    # Create a rotation that aligns Z axis with 'forward'
    align_rot, _ = R.align_vectors([forward], [[0, 0, 1]])
    # Add a twist around the forward axis (drill bit effect)
    twist_rot = R.from_rotvec(theta * 10 * forward)
    # Compose the two rotations: first align, then twist
    rot_obj = twist_rot * align_rot
    rot_matrix = rot_obj.as_matrix()
    # Build 4x4 transform
    tf = np.eye(4)
    tf[:3, :3] = rot_matrix
    tf[:3, 3] = position

    tf_axial = tf.copy()
    # Project 'forward' onto RS plane (zero A component)
    f_proj = forward.copy()
    f_proj[1] = 0
    # Directly align projected vector to Z axis (no need to normalize)
    axial_rot, _ = R.align_vectors([f_proj], [[0, 0, 1]])
    tf_axial[:3, :3] = axial_rot.as_matrix()

    tf_sagittal = tf.copy()
    # Project 'forward' onto SA plane (zero R component)
    f_proj = forward.copy()
    f_proj[2] = 0
    # Directly align projected vector to X axis (no need to normalize)
    sagittal_rot, _ = R.align_vectors([f_proj], [[1, 0, 0]])
    tf_sagittal[:3, :3] = sagittal_rot.as_matrix()

    tf_coronal = tf.copy()
    # Project 'forward' onto AR plane (zero S component)
    f_proj = forward.copy()
    f_proj[0] = 0
    # Directly align projected vector to Y axis (no need to normalize)
    coronal_rot, _ = R.align_vectors([f_proj], [[0, 1, 0]])
    tf_coronal[:3, :3] = coronal_rot.as_matrix()

    tf_msg = pyigtl.TransformMessage(tf, device_name='ToolToRAS')
    server.send_message(tf_msg, wait=True)
    tf_msg = pyigtl.TransformMessage(tf_axial, device_name='ToolOnAxial')
    server.send_message(tf_msg, wait=True)
    tf_msg = pyigtl.TransformMessage(tf_sagittal, device_name='ToolOnSagittal')
    server.send_message(tf_msg, wait=True)
    tf_msg = pyigtl.TransformMessage(tf_coronal, device_name='ToolOnCoronal')
    server.send_message(tf_msg, wait=True)
    # Since we wait until the message is actually sent, the message queue will not be flooded

However, that seems a little excessive in practice, as I’d also like to re-use the Stabilize processing mode via TransformProcessor for low pass filtering of noise from navigation tracking. I tried playing around with the various other processing modes provided by TransformProcessor to collocate all aligned transform computation within Slicer 3D fronted itself, simplifying the use Stabilized transform trees, but couldn’t figure out how to replicate the same alignment as above in python.

Any suggestions using TransformProcessor, or adding such a mode to VolumeResliceDriver?

2

There is no explicit representation of a 5DOF transformation in Slicer, so I wouldn’t update either VolumeResliceDriver or TransformProcessor. I think 5DOF transformations are relatively rare, the only application I can think of is Aurora needle trackers. I would just observe the incoming “5DOF” transform and run a script like the one you have to convert it to a regular transform every time it updates. And use the modules with that recomputed transform. Is there anything that prevents you from doing that?

1 Like
3

They are expressed in the Bullseye View Mode for the external Viewpoint extension in SlicerIGT:

image
image584×775 57.2 KB

But, yeah, that’s more of an end use case rather than an standard slicer data node type.

Wow, I figured it’d be more common, like for elongated tools and volume path views in general.

Suppose I could bundle this into the our existing python extension, but was hoping this would already have been available out-of-the-box using a combination of existing IGT extensions. I mostly just wanted to offload this kind of 5DOF computation until after the TransformProcessor has had the chance to apply a low pass noise filter to the main child transform, thus avoid any round trip data flows out and in of Slicer 3D, like with the pyigtl example above.

As an aside @ungi , would you have any suggestions to avoid the jarring discontinuity of the current approach when the z axis of the tool lines up with any RAS axis of the volume? The relative rotation around the 6th axis for the 5DOF transform flips 180 degrees when passing through such Euler angle singularities.

Perhaps there’s better public examples using quaternion for computing 5DOF projections?

Suppose I could borrow from the workaround applied in Viewpoint’s Bullseye View Mode:

4

Example discontinuity of the current approach shown in inline sagittal slice view:

5

We usually set the up direction of a slice view from a 5D transform using Python scripting. See a full example in thebscript repository: Script repository — 3D Slicer documentation

would you have any suggestions to avoid the jarring discontinuity of the current approach when the z axis of the tool lines up with any RAS axis of the volume?

The exanple in the script repository takes care of this.

Probably we have not been motivated enough to add this feature to volume reslice driver module because the ideal behavior very application-specific. In each clinical workflow that we developed we ended up using slightly different method. But maybe you can convince us to add a few common strategies.