Markups toolbar complexity for simple tasks

[jamesobutler (James Butler)]

There have been recent changes to how users create and place markups using the toolbar. These changes have been written up in the following post.

Changes to the Markups Module

Several updates to the Markups module and the addition of a Markups toolbar were added in commit: ENH: Improve control point state management and add markups toolbar · Slicer/Slicer@09d12ae · GitHub and are available in the latest preview version. The motivation for these changes was to add support for unplaced or missing control points. These states make it possible to create landmark templates for efficient placing of pre-named points, which is a common feature in other landmarking software. …

It appears to be a great change to support easier functionality when using the markups Fiducial object for registration purposes, however it comes with a side effect of additional complexity for other markups such as “Line” which are primarily used for more simple measurement tasks. I hope with this thread we can brainstorm ideas to support the new functionality for the markups Fiducial object, but help make the other markups simpler to use again.

In previous versions of Slicer, to place a line (aka ruler) node, ROI node, angle node, etc you would go to the mouse mode toolbar and click in the menu to add a new node for the object and also enter placement mode. You would then place your 2 or 3 points, maybe use your mouse to adjust a point, view your measurement or accept your ROI for cropping and then be done. A fairly simple task with not a lot of interaction involved and when interaction is needed you can just use the mouse to move a position or right-click to delete or whatever. To place a second line I could simply click on the place button which now had a default action for the line node and create and place another object.

However, now to follow the same pathway I must, toggle the markups toolbar from the mouse mode toolbar, go over to the markups toolbar, find my line node button in a large group of new widgets, then go to the slice view to place my line. First, a minor point in that the movement of the mouse across the screen is a bit annoying as default behavior to go from one toolbar to another. Yes, the toolbars can be rearranged, but by default is not as simple as before.

Second, the bigger issue is that there are all these new buttons in the new markups toolbar that are intimidating when going to just place a line node. It gives the impression as though I will need all these options for line node placement when really that is not the case. Node selection, color selector, place button, trash can to delete last placed control point or to unset control points, or other options for visibility and lock states. These are for the most part not needed at all for simple markups for measurement purposes. I do see the value for these for when dealing with Fiducial objects and manipulating/adding control points to various landmarking template lists across Fiducial objects, however they add additional complexity for simple tasks of placing a markup for quick measurement.

How can we make markups for measurement purposes simple without showing a lot of options that can be intimidating to users? There seems to be a split in the need of options between the “Fiducial” objects need for position status (unset, set, restore, skip), switching between nodes, setting colors to various groups, etc and the simple needs for less complex workflows of the other markups.

[muratmaga]

While I agree that there are things we need to streamline with the new markups toolbar, I am not sure if the behavior is all that different, or complicated for simple things like measuring lines, angles etc.

This is how it is in stable

This is how it is now with new toolbar

To me that doesn’t seem like a change that should confuse a user who is used to the old way. And it won’t matter for new users, because that’s the interface that they will learn.

The change of place mode button as a toggle is an intentional change to make sure that long time users of Slicer are aware the things are different in terms of markup placement (that is there is now a specific toolbar to use).

[jamesobutler (James Butler)]

The main issue for line node creation using toolbars is that now I have to show a toolbar (“Markups toolbar”) that has all these new options which is intimidating when I just need to use one of them for line node (create line node button). It’s the fact that there are more widgets to navigate through for a simple task like measuring some with a line node. It’s the same type principle of why advanced sections exist in a collapsed state to avoid viewing things that are typically unnecessary. For the case of line nodes and other simple measurement markups, all the other options in the new markups toolbar are unnecessary.

[muratmaga]

@jamesobutler

The main issue for line node creation using toolbars is that now I have to show a toolbar (“Markups toolbar”) that has all these new options which is intimidating when I just need to use one of them for line node (create line node button).

I respectfully disagree with that. it is the same set of icons as before on the left hand of the toolbar. We simply rearranged in a horizontal view so all markups types are visible. We can change this back to dropbutton as it was before, I don’t have any issue with that. The right handside of the toolbar contains a mix of old and new icons (for the new clear, edit functionality). If a user never used any those before, I suspect that they will pay no heed to them.

The previous interface was simple, but if you are only going to do one type of markup node (perhaps lines, or angles). Once you start using curves and fiducials together, there is really no way to know what node you are editing/creating, what others are etc. That required users switching to Markups (or data) module and use half of those features there for simple things (like color change, rename a node etc), and we are back to users switching to different modules for simple tasks.

[jamesobutler (James Butler)]

Yes, user may not need any of those new widgets in the markups toolbar which is the problem. Compared to the old method, creating lines, angles, etc measurement markups is not as simple as before because you have to search through more things that you’re not going to use.

@muratmaga

The previous interface was simple, but if you are only going to do one type of markup node (perhaps lines, or angles). Once you start using curves and fiducials together, there is really no way to know what node you are editing/creating, what others are etc

This brings back up the main topic I’m hoping to address which is how to keep simple measurement markups simple without other widgets as distractions, while also supporting the cases you have detailed where some markups do require more interaction for things like registration.

[lassoan (Andras Lasso)]

Now that we have right-click menu for markups with many actions available there (and you can open the full Markups module menu from there), I agree that you could manage measurements by using the place mode button and Markups module (without showing the toolbar). But we had a lot of issues due to the dual meaning of the place button: create new markup / add point to existing markup.

OsiriX has a single place button for adding new markup and adding new points (same way as Slicer had it before), and as far as I can tell, it is simply not possible add points to existing curves (it does not have point lists).

Using only the mouse mode toolbar, how would you distinguish between adding new points to an existing markup or create a new markup? How would you indicate that you have placed enough points (in the current curve or fiducial list) and you would like to start a new one?

I agree Mouse move between the left side of the screen and the viewers is not good. We could make things easier by adding keyboard shortcuts and/or making the toolbar vertical-friendly (do something with the node selector) and snap it to the right side of the screen by default.

We will probably need to have a docking widget for fiducial placement that can be used while being in a module (so that the user can see point placement progress and choose the next point to place in any module, without the need to switch to the Markups module), similarly to what is done in Orthodontic Analysis extension:

We should think about how to distribute features between the mouse mode toolbar, markups toolbar, measurements/landmarking docking widget, and markups module, which fulfills requirements of all use cases and respects “makes simple things simple and complex things possible”.

To facilitate this discussion, would you mind briefly summarizing your use cases (what would be the ideal way the features would work)?

Just to avoid having use cases spread across several posts, as an experiment, let’s try to use the Slicer GitHub wiki for this. Add your use case to this page. In the end we’ll see if we want to leave the use cases there or bring the result back here and remove that labs page.

[jamesobutler (James Butler)]

Yes, I will provide ideas and use case information to the above bolded items. I have currently been brainstorming and trying to organize thoughts on the topic in my notes so I can provide clear and detailed information. A very short summary of what I will be providing is that markups for measurement follow the “Create and place” action with little to no editing after the original interaction of placing. When there is interaction after the initial interaction has ended, points are moved by mouse left-click, more infrequently deleted by mouse right-click menu option and more infrequently selecting curves to ctrl+mouse left-click to insert additional points.

[mikebind (Mike Bindschadler)]

I added a set of the use cases which jumped to mind for me on the wiki page. For my purposes, the new toolbar is great and much easier and clearer to work with than the old version. However, I only very rarely use the simple ruler or angle measurement tools. I use the fiducials, open curves, planes, and ROI’s all the time, and the clarifications around when a new node will be used for a new point placement vs an additional point to a current node, as well as the improvements facilitating landmark templates, are very helpful.

[jamesobutler (James Butler)]

I’ve added use cases to the wiki labs page. Explains about “Quick measurements” where markups are created and points placed in one interaction (“Create-and-place”). Markup control points are usually not removed or added after the initial interaction. Since the markup is easy to redo, it makes sense to just recreate from scratch. Often deleted with right-click menu.

[jamesobutler (James Butler)]

Linking this thread below which began the discussion about how to improve knowledge of whether a new markups node was created or new control point added to a markups node.

Markups node creation icon is confusing users

Currently same icon is used to create both a new fiducial in a list [1 below], and a new fiducial list [2] in the Markups module. This behavior is confusing to the users. I propose to modify the node creating one slightly to make it less similar. It might have visual clues to what it does (e.g., add three dots, have word Node/List on it etc). [1] [image] [2] [image]

Summary:

• It was confusing if you “place a fiducial” if that means you are placing a node or placing a control point for a node because the vtkMRMLMarkupsFiducialNode is a list of control points.
• Some issues brought up included the overall size of an updated markups related toolbar as the default state is what users will use first and will not bother to customize. Space concerns started the topic of how best to support more multi-module functionality availability at the same time. This includes how to have more Markups module functionality availability at the same time of using another module. Ribbon interface was mentioned.

^ This topic is similar to what @pieper mentioned in a weekly hangout about how best to support multi module use cases when space concerns are being brought up. The markups toolbar with a node selector uses more space. It is trying to add Markups module functionality to the toolbar as a solution to support that Markups functionality while actively in another module. Will this spark more modules to just add their functionality in a toolbar as a solution for multi-module workflows?

A lot of the vtkMRMLMarkupsFiducialNode use cases are paired with other modules for landmarking purposes. So a multi-module solution of functionality is needed. However vtkMRMLMarkupsLineNode is primarily a single module workflow as the distance measurement is provided by the markups module and can be manipulated in the slice views without a real need of viewing the module.

[lassoan (Andras Lasso)]

@jamesobutler

Will this spark more modules to just add their functionality in a toolbar as a solution for multi-module workflows?

Adding a toolbar does not make sense for all modules but only for those that provide features that are intended to be used while in another module. Current examples are Sequences, Scene Views, Virtual Reality, and Markups.

I would hesitate to make multiple module GUIs displayable at the same time, but allowing multiple toolbars seems like a good tradeoff.

[jamesobutler (James Butler)]

@lassoan Are we at a time now based on the use cases to brainstorm towards making markups actions simple for measurement purposes?

[lassoan (Andras Lasso)]

Yes, we can assume that we won’t get anymore use case descriptions. Probably the best would be to discuss this during a call. Would all interested people be available to join the next Slicer weekly meeting (Tuesday at 10am)?

[jamesobutler (James Butler)]

Yes, unusually I can actually attend this next meeting if we want to discuss more about this topic then.

[smrolfe (Sara Rolfe)]

I will be available to attend next week also.

[ezgimercan]

Following up this, on adding a measurement toolbar (or annotation toolbar): This is a functionality most DICOM viewers have. They even have text, arrow, ellipse, circle, pen options - but instead of markups, these are annotations (kinda like Zoom or video call annotations on the screen). With the exception of ruler and angles, they don’t return any measurements. I think a user with a medical background would expect to see this in Slicer. It is a very common thing for doctors to want to measure something to just see the number. “Let’s see how wide the eyes are, hmm, 2.4 cm is wide for this age…” kind of situation.

See the web-based viewer our hospital uses:

Previous mouse mode button with all markups nodes available under the expandable menu provided this functionality for users who didn’t care about the line markups node as much as being able to see the distance. With the current Markups toolbar, it is still possible, and as a heavy user of fiducials, it is still very straightforward and easy for me. But I understand the complexity this toolbar might present for a new user. If the idea is to simplify the use of Markups for some users, I like the option of adding a measurements toolbar - which uses the functionality of Markups and still creates lines, ROIs etc. in the scene but hides all the extra functions.