Interactive slice intersections

Until now, slice intersection navigation was based on hotkeys like Shift+drag for moving the 3D cursor, or Ctrl+Alt+drag&drop for rotation of the slice planes when slice intersections are shown.

We added a new feature to enable interactive slice intersections. This feature will enable to move the slice intersection point or rotate the slice planes using only the mouse, by clicking and dragging the slice intersection lines in the 2D views. When hovering over the intersection lines, the mouse cursor changes to indicate which action is available for the user: translation or rotation. Ordinary keyboard-based controls are also available using this new interactive mode.

This feature can be activated and customized in the ViewersToolbar once the ordinary slice intersection mode has been activated. Also, translation and rotation actions can be enabled or disabled using the Interaction Options in the ViewersToolbar.

See video below:

Some useful links:

The feature is available in the latest Slicer Preview Release.

Any feedback and suggestions are welcome.

David

10 Likes

It is Fantastic… I love it, thank you very much. Maybe this function could work with the crosshair tool too, and allow 3D visualization of the intersection.

1 Like

Wow! This looks great!
:clap: :+1:

Is there a way to have the slice intersections be interactive, but also still intersect? Currently, when in interactive mode, they are in a crosshair mode and not actually intersecting. Slicer’s Crosshair object has both a “basic crosshair” and “basic + intersection” option.

Interactive is OFF:
image

Interactive is ON:
image

The current implementation can be adjusted to show the intersection. There is a constant in vtkMRMLSliceIntersectionInteractionRepresentation that can be modified to “ShowIntersection”. This will show the intersection of the lines, while maintaining the rest of functionalities.

By default, we decided to keep the intersection hidden. You could change the visualization mode by changing the value of this constant in your custom build.

However, we may consider adding a property in the vtkMRMLSliceDisplayNode so that this visualization mode can be easily modified by the users. We’ve already added some custom properties in the vtkMRMLSliceDisplayNode to control the visibility and to enable/disable the rotation and translation actions for the interactive mode. We could add something similar to control the visualization mode, defining if the intersection is shown or not.

1 Like

This is certainly a great addition making reslicing easier for new comers.

I’m wondering about the feasibility of a small enhancement :

  • Add more buttons in the ‘Crosshair selection’ toolbar
  • One button to activate intersection
  • One button to activate intersection interactively

Currently, the drop down menu allows to activate intersection, and we have to open the menu again to choose interaction. More buttons would allow a one-step switch.

It’s not compelling, just a small usability addition.

1 Like

Yes @dgmato would you be able to make the following changes to have the options like the current crosshair options?

  • “No crosshair” is like the “Slice intersections” checkbox being unchecked
  • “Basic crosshair” would be “Slice intersections” checkbox checked, with the crosshair spacing, but with interactions off
  • “Basic + intersection” would be like the current option where “Slice intersections” checkbox is checked, with interactions off

The “Interaction” checkbox would then only toggle whether interaction is on/off and would not make any display changes to slice intersections. Display of the slice intersections such as whether the lines comes to a cross or have an open crosshair would not change and same goes with slice intersections line thickness would not change either when “Interaction” is toggled (line thickness is currently changing which is odd).

image

1 Like

@chir.set @jamesobutler I agree with you that the toolbar selection could be improved. Currently, you need to open the menu twice to activate the interactive slice intersections mode, which may not be convenient.

If I understood correctly, your idea is to add three more options to control the visualization mode of the slice intersection mode. These three options would substitute the “Slice Intersections” checkbox. Then, we would have one check box (“Interaction”) to activate/deactivate the interactive mode, and the “Interaction Options” to enable/disable the translation and rotation actions during interaction. Right?

I like this idea proposed by @jamesobutler. I think it would be pretty intuitive for the users. Also, I really like the icon you showed in this post.

The current design of the toolbar options was proposed by @lassoan, who suggested following a similar approach to the markups interactivity configuration (see discussion here). What do you think @lassoan?

It seems like there is desire from some groups to have the open crosshair display type, while others prefer the lines actually fully intersecting. Due to this requirement I believe the options have to be like what was already done for the “Crosshair” object type. If just one of these options was all that was needed, then the simple checkbox could be done, but that doesn’t seem to be the case here.

Yes that is correct.

I think this similar duplication between “Slice Intersections” functionality and the “Crosshair” object functionality is what I was discussing at in another thread. This post (linked below) from @cpinter reflects on this topic of maybe “Slice intersections” is truly replacing the “Crosshair” object because it’s hard to see a reason to have duplicate options for each here in this menu.

Not exactly. The crosshair toolbar has only 1 button currently. I’m thinking of a second toggle button that would activate/deactivate intersection, and a third toggle button that will activate/deactivate interaction (and also activate/deactivate intersection by necessity).

If the second button is deactivated, the third one should be deactivated.

If the third one is activated, the second one should be activated.

The drop down menu does not have to change.

The crosshair toolbar would contain 3 buttons in all, instead of one (which is currently like a sub-menu in a menu bar).

I agree that we should add a separate toolbar button for slice intersection display. The intersections could be displayed by a single click and interactivity could be enabled by 2 clicks. Configuration of the slice intersection type could be also added in the menu of that button.

So, we would end up with two buttons in the crosshair toolbar:

  • Crosshair (toggle)
    • No jump slices / Jump slices - offset / Jump slices - centered
    • No crosshair / Basic crosshair / … / Small basic + intersection
    • Fine crosshair / Medium crosshair / Thick crosshair
  • Slice intersections (toggle)
    • Interaction (toggle)
    • Interaction options (submenu): Translate (toggle) / Rotate (toggle) – we can change this design but it should be consistent with how we configure interactions for markups (in view context menu when clicking on a control point)
    • Full line / Skip line crossings – in the future there could be an option to show the line for the infinite slice plane (don’t shorten the line when a slice is zoomed in)
    • Fine / Medium / Thick – not needed now, just could be added in the future

In the future we could add an application settings to set default slice intersection mode (interactive or non-interactive). I would not add it yet, because the feature is still somewhat immature: it is not compatible with Segment Editor (due to Segment Editor not using widgets and so there are two components competing for viewer events). For now, users can add a Python code snippet if they want to enable interactive slice intersection by default.


About removing crosshair. I would love to see any simplification and removal of unused features, but I don’t think we can remove the crosshair, no matter how sophisticated slice intersections become. Crosshair and slice intersections are related, as both display lines and the crosshair can make slices move (and therefore move slice intersections). However, the crosshair has some important properties:

  • there is always one single crosshair position in the entire scene (while there may be 0 to many slice intersection points)
  • a crosshair can be always specified (even when there is only a single slice view, or there are no slice views just 3D views)
  • the crosshair can be used to cross-reference a position across slice and 3D views; or between 3D views
  • the crosshair position can be always set to the current mouse pointer position (while slice intersections are preferably drag-and-dropped, because if you have many slice intersections it may not be clear which intersection the user intends to move)
2 Likes

Separating the two features seems to have a consensus.

Yes I assume there are users who use crosshairs and would like to keep doing so. I, however, am still a bit confused about how it works (or supposed to work), and what is its purpose, i.e. how it is intended to be used.

  • It cannot be drag&dropped, it can only be moved using the Shift key (is this true btw?). I really would expect that if it is an important “landmark” then it can be moved without shortcuts.
  • If you drag the slider in a slice view, then there is apparently no way to navigate back. Again, if we consider the crosshair VIP position in the scene, jumping back should be simple, but at least possible.

So to me it seems like a bit obscure feature for select users, and I imagine that if I had to highlight points in the scene, using fiducial points are easier (and they support both above features in a simple way). The only thing fiducials cannot do is moving the VIP one with the Shift key.

The main use of the crosshair node are the followings

  • Show a single point in all views for cross-reference (regardless of slice views exist or they intersect): you use it by toggling the crosshair button and moving the crosshair using shift+mousemove
  • Provide position for the Data Probe or any other module that wants to get a current mouse cursor position.
  • Move slice intersections: enabled by default but if you think that with the new slice intersections feature it is no longer needed then it could be made configurable in the application settings.

These are all very commonly used features. We could say that they are somewhat “obscure” because it is not obvious how all these features are related to the crosshair. There is documentation (user, developer, and quite a few examples in the script repository) - would adding more details help?

So, if I am understanding correctly, the crosshair location’s real function is to serve as a singleton “currently selected location”. This idea cannot be replaced by the concept of “wherever the slices intersect” because there may be no such place or many such places since the number and orientation of slices can be arbitrary. However, in the other direction, it is always possible to move existing slices such that the slice plane contains a “currently selected location” while maintaining their existing normal vector.

One other thing I would like to note is that moving slices (via shift-mousemove) only moves slices in the same ViewGroup (as defined in the layout). This is an important feature because sometimes you want leave one group of slices in a particular position while you search through another group (e.g. while looking for matching fiducial points in two image volumes). Currently, this is handled by only moving slices in the same ViewGroup as the view or slice over which you are shift-mouse-moving. I just wanted to mention this use case because it is a bit of an exception to the idea that all slices move in response to the crosshair postion.

I only very rarely use a visible crosshair, generally preferring the more visually informative colored slice intersections, but I use shift-mousemove all the time, which I now understand is using the crosshair node under the hood.

1 Like

Hello, David.

For the translation, I wondered how it compared with the existing functionality of pressing Shift while moving the mouse (hovering) across one of the Slices. I usually do this with the crosshair turned on.
A separate discussion seems to have been created on that topic.

The rotation is a cool feature. The question I had was about reverting to axes lining up with the original (DICOM/RAS) coordinates. After some rotation has been performed, how should the user return to the ‘default’ alignment?

—DIV

In each slice view, you can select ‘Axial’, ‘Coronal’, or ‘Sagittal’ to return to world RAS coordinate axes.

image

To return to the IJK image volume array axes directions, you can click the ‘Rotate to volume plane’ button:

image

1 Like

I’ve issued the following PR with work for separating these two things into their own QToolButtons.

1 Like

Thank you dgmato. It’s very interesting and useful. Without this useful feature, my 3D slicer’s crosshair has only almost useless features such as Fine crosshair, Medium crosshair, Thick crosshair :)) but no rotation and translation!
However, I have a suggestion. Please kindly complete your post by clarifying how this feature can be activated and customized in the ViewersToolbar, and how the ordinary slice intersection mode can be activated. Also, how translation and rotation actions can be enabled or disabled using the Interaction Options in the ViewersToolbar, please? :slight_smile: