New transform widget for Slicer

Development
1

Hi devs. Like I mentioned on the Slicer meeting. I would be interested in contributing a transform widget similar to the one used in the following video: Virtual surgical planning using a DCIA free flap for mandibular reconstruction - YouTube

As you can see in the video next 60 seconds the iliac crest is registered to the mandible successfully. This is the software 3D Systems use for planning this kind of surgery. And I think this widget would be useful for making this planning possible in Slicer also.

3DSystemsTransformWidget.PNG

The widget has two modes (possibly activated with hotkeys):

  • The widget reposition mode, where you move the frame of the widget without transforming the mesh
  • The widget transform mode, where you are able to rotate the mesh around the widget’s origin or translate the mesh by moving the widget

The widget allows traslations that are:

  • Linear, following the arrows of the frame
  • In the camara plane, moving the origin of the frame. (*)

The rotations can be:

  • Using the rings corrresponding to each arrow of the frame (rotation around that axis)
  • Using the ring that has and axis always parallel to the camera direction of view. (*)

Both (*) I consider very useful because those are the only movements used in the registration shown in the video.

I would appreciate comments or suggestions regarding/guiding the implementation.

My objetive would be to make the widget as similar as possible to the one shown in the video because I consider it has great usability.

1 Like
2

While such widget is never the best tool for alignment (it is very inaccurate and slow), I agree that it would be nice to have it available in Slicer.

We already have an issue to track this task (Transform Widget: easy enhancements for more user-friendly manual registrations · Issue #2579 · Slicer/Slicer · GitHub), but I think you summarized the main requirements well.

We already have a transform widget but it has a few important limitations:

  • the center of rotation is not adjustable
  • there are no arrows to limit translation/rotation to a single axis at a time
  • it uses a plain VTK widget instead of being based on vtkMRMLAbstractWidget

There are many advantages of vtkMRMLAbstractWidget over basic VTK widgets, including support for multiple views, efficient (updates itself directly from MRML), built-in methods for focus management (as opposed to VTK’s complex and limited picking manager), support for interaction in multiple contexts (e.g., manipulating with the mouse on the desktop and using two virtual reality controllers in an immersive view), and fully customizable keyboard and mouse shortcuts.

You could update vtkMRMLLinearTransformsDisplayableManager3D to work similarly to how vtkMRMLMarkupsDisplayableManager works. You could create the vtkMRMLAbstractWidget based widget by cloning and modifying vtkBoxWidget2, but since we probably don’t need the box, I think it would be better to start from one of the markup widgets. Markup widgets already have handles for rotation and translation, which should be usable for transforms, too. You can ask advice from @Sunderlandkyl about how to use these arrows in your widget.

Instead of having a reposition and transform mode (state), I would recommend to just use different keyboard shortcuts to translate/rotate the transform axes or change the transform.

Translation along the arrows or the camera plane is already implemented (dragging in the center handle moves in the camera plane; dragging the arrows translate along that axis).

Rotation: current rotation arrows only allow rotation along the widget axes, but I agree that it would be nice to add a rotation around camera axes option, too. It could be either by adding an outer ring or by adding a button somewhere that would toggle between widget an screen axes. Probably the outer ring is a bit simpler to implement and use.

It is important to use the same translation/rotation arrows implementation in both markups and transforms, otherwise we would duplicate maintenance effort. I like how the widget looks in the video, so if you can make our current arrows prettier in any way then that’s great.

3

I think the first step would be to add these actors to the vtkSlicerMarkupsWidgetRepresentation: NormalToRedSliceRotationActor, NormalToYellowSliceRotationActor, NormalToGreenSliceRotationActor and NormalToCameraPlaneRotationActor

I don’t know how to selectively render this actors so that NormalToRedSliceRotationActor is only rendered in the RedSliceView and so on.

Is this approach okay @lassoan? Maybe after I have this working I can work on the transformsWidget

4

Each view has its own widget representation object, so you need to add only one more actor for rotating around the view normal.

5

Last project week I started moving the interaction handles from Markups and making them more generic, and adding them to Transform nodes. This is the branch that contains the changes: GitHub - Sunderlandkyl/Slicer at interaction_display_manager.

1 Like
6

Hi Kyle. I’ll look into it when I have the normalToViewRotationHandle ready for markups.

@lassoan these are the functions I don’t know how to rewrite because now there are two actors:

void vtkSlicerMarkupsWidgetRepresentation::GetActors(vtkPropCollection* pc)
void vtkSlicerMarkupsWidgetRepresentation::ReleaseGraphicsResources(vtkWindow* window)
int vtkSlicerMarkupsWidgetRepresentation::RenderOpaqueGeometry(vtkViewport* viewport)
int vtkSlicerMarkupsWidgetRepresentation::RenderTranslucentPolygonalGeometry(vtkViewport* viewport)
vtkTypeBool vtkSlicerMarkupsWidgetRepresentation::HasTranslucentPolygonalGeometry()

Here is my branch.

7

Hi @lassoan and @Sunderlandkyl

Can we talk about the implementation plans of this feature? I really want learn some more C++ and VTK, and I’ll have time starting next month to work on it.

I already have a buggy branch that adds in-plane rotations to the interactiveHandle of the markups in both 2D and 3D views. I could start from here.

Also just to not cross-post, here is the issue about this

8

Sure, let me take a look a the existing branches and come up with a plan to move forward. I’ll see how the work could be divided up.

1 Like
9

Here is another implementation of a transform widget using a cube (apparently for rotations) :slight_smile:

1 Like