Redesigned markups allow us to add a lot of new features, and we need to ensure that these new features are conveniently accessible via keyboard shortcuts and mouse gestures.
Since number of mouse buttons and modifier keys are limited, one way of making more features accessible is to introduce more mouse modes. In each mouse mode, gestures could be mapped to different actions.
As a first step, we’ve introduced “Window/level” mode in latest nightly version, to prevent accidentally changing volume window/level (left-click-and-drag only changes window/level in the window/level mouse mode) and also to allow more sophisticated window/level adjustments (left-click-and-drag works as before; Ctrl + left-click-and-drag to highlight a region and optimize window/level for that; pressing Escape or right-click cancels the operation):
What mouse modes would you like to have (instead of/in addition to, binding it to some complicated keyboard&mouse gesture)?
node select (we could select/unselect objects by clicking on them)
node translate, rotate
slice intersection move/rotate (in recent Slicer versions you can rotate intersecting slices if you show slice intersections and use Ctrl+Alt+Left-click-and-drag - but many people will not discover this)
split view zoom, pan, rotate, slice selection to a few different modes
I just wanted to say this is welcomed improvement. The group I work with has implemented some things in the past that this now addresses.
Prevent accidentally changing volume window/level is a major one. We had always locked it and made the current volume lock/unlock window level with a keyboard shortcut.
Also we have moving slice intersections done by mouse hovering over an intersection and left click and dragging it to a new position. We implemented this because we observed users trying to do this instead of mouse scrolling in other slice views as a way of changing slice intersection shown in another view. Also I don’t think they knew about the shift+mouse move ability to move slice intersections.
Since slice intersections are ow shown by a widget, too, we can easily implement moving the center or rotate it by drag-and-drop (I think it’s already implented, we just have not activated it in Slicer master version).
One additional ww/wl mode to consider: similar to the rectangle, draw a line and use that to define the range for the ww/wl. For example if you really wanted to focus on the range of intensities in the skull, you could draw a kind of wavy curve through the bone (I had done this in a system many years ago and it was really handy).
Also, ideally these roi-based ww/wl adjustments should hot-update on every mouse move.
I’ve seen many users accidentally change the window when trying to change the slices offset as in ITK-SNAP or MITK. I think enabling this by default could be useful, or maybe adding a button for it.
Quite unrelated, but I think the automatic window settings algorithm should be improved. I almost always have to adjust the contrast, no matter which image I’m loading. I think the automatic contrast adjustment of ITK-SNAP (Ctrl + J, I think) works great. I once looked into it to try to implement it for Slicer, but it all seemed out of my league and I finally gave up. Happy to participate in a potential effort, though.
I’ve had a look at how ITK-Snap computes window/level (AbstractContinuousImageDisplayMappingPolicy::AutoFitContrast()) and it seems that it just uses linear scale between 0.1%-99.9% range of the image. We do the same in the regional auto-contrast (Ctrl + Left-click-and-drag in Adjust window/level mode). The limitation is that it always creates relatively low-contrast images (does not try to determine what is the relevant signal in the image).
Slicer’s auto window/level algorithm performs bimodal analysis of the image - it assumes that image consists of signal corrupted by noise and determines a window/level that optimizes visibility of the signal. In general this creates images with much better contrast, but it may make the image contrast too high if the image histogram’s first local maximum is not the actual signal’s peak.
Example: MRBrainTumor1 sample data set
ITK-Snap auto-adjust contrast produces very dark, low-contrast image:
So, I think the current options for manual window/level adjustments cover most use cases quite well. Of course, there is always room for improvement, for example what may be missing:
loading of default window/level preset from DICOM and other file formats that can store window/level
setting WW/WL to multiple volumes at the same time could be useful, but I don’t know where and how on the GUI we should offer this feature (maybe it could be added as a menu action in the subject hierarchy tree)
If anybody would want to see improvements in these then we can discuss it further here and/or add an item in the issue tracker.
slicer’s bimodal analysis does almost always do a good job for me, but I know @hjmjohnson told me years ago that when he loads masked images he gets bad result (e.g. skull stripped MR neuro). This is because the bimodal assumes the two modes of the histogram will be air (ignored) and tissue (used to define the ww/wl). For masked images the air is typically all 0, so the histogram is not really bimodal and the contrast is somewhat arbitrary. We never dug deeper at the time, but maybe now we should and probably a simple heuristic could solve this case.
Also I thing context menu option to pick a strategy or just having a hotkey to cycle through strategies could be good.
window width / window center values from DICOM are loaded and applied, at least for scalar volumes, If there are multiple they will all be available. These are in the window/level presets portion of the Volume Information section of the Volumes module.
Just in case this is useful:
I’ve been doing visual inspection of a lot of T1-weighted images recently. Most of the sequences are FSPGR 3D, FSPGR BRAVO or MPRAGE. Typically, the automatic windowing is good for BRAVO and MPRAGE and bad for FSPGR. Here are some screenshots of Slicer and ITK-SNAP for FSPGR 3D and FSPGR BRAVO of the same patient:
Thanks for the follow-up. I agree that FSPGR_3D volume is displayed overexposed with Slicer’s global auto WW/WL setting (double-click on the image in WW/WL adjustment mode), while it looks nice with the area-based auto WW/WL (ctrl-left-click-and-drag in the middle of the image and drag it to cover the entire image).
It is because the global WW/WL setting detects the blanked out region in the image (red) as background and tries to improve visibility of the noisy background region (orange) by making it brighter.
We could probably tune the histogram analysis to make it a more robust against small blanked out regions. Or maybe allow users to choose between automatic WW/WL strategies (stretch WW to 0.1-99% range or to auto-detected range).
Wouldn’t it be reasonable to completely ignore 0 intensity by the auto WW/WL setting algorithm? I guess this could cause problems when loading segmentations as grayscale images, but in other situations 0 can probably be ignored safely, since some level of noise will be present in the real image region. In MRI I believe those are not blanked regions, they are probably 0 because of how the image was reconstructed.
I’ve created a script to test Slicer’s current (bimodal analysis based) auto window/level method and compare it to fixed percentile based method, on all Slicer sample images.
Description of methods used for generating the images
baseline: current method in Slicer
hist-0.1-99.9: minimum at 0.1th percentile, maximum at 99.9th percentile, no expansion (this setting is used by ITK-Snap)
hist-1.0-99.9: minimum at 1th percentile, maximum at 99.9th percentile, min expanded by 10%
hist-1.0-99.0-x0.10: minimum at 1th percentile, maximum at 99th percentile, min expanded by 10%, max expanded by 10%
hist-1.0-99.0-x0.20: minimum at 1th percentile, maximum at 99th percentile, min expanded by 10%, max expanded by 20%
The fixed percentile based method is implemented in vtkImageHistogramStatistics: window min/max is set at a certain percentile of the histogram, optionally extended by a fraction of the window size.
From the results it is clear that the current auto window/level method is not optimal. Suppressing the first peak in the histogram is not necessarily a good idea for general use, as important signal may be in the first (“noise”) peak and it is hard to reliably detect boundaries of the “signal” peak. It would probably better to switch to the simple percentile method.
Thanks for doing this analysis @lassoan! I’m fine with changing to one of the histogram methods. Matching ITK-Snap could make sense if nothings obviously better.
Also useful would be to give users an easier way to discover window/level settings and presets, probably through a context menu on the slice viewers or similar.
Methods 3 and 4 look best to me. Do you know why the CT scans “look good” (air and background are black) only sometimes? For example CT-cardio vs CT-chest.
It mainly depends on what intensity the background (outside the cylinder) is set to. Background in CTACardio is -1024, in CTChest it is -3024.
Method 3 (hist-1.0-99.9) seems to be the best to me, too, as it produces somewhat lighter images than method 2. However, method 2 (hist-0.1-99.9) has the advantages that:
it is symmetric (so, for example, if you invert the image you get the same min/max values; and it works the same way regardless the background is bright or dark)
it is the same setting as ITK-Snap uses
I agree, it should be more discoverable. Adding menu (subject hierarchy menu items and some others, such as mouse mode) is on my list.
I’ve submitted a pull request for using fixed 0.1 and 99.9 percentile for setting window/level automatically (Method 3, hist-1.0-99.9, same as ITK-Snap).
This concerns rotate in 2D views. It might be useful to have handles at the ends of each intersection line, that would rotate the current view only, or the corresponding view only (red handle for red view), without affecting the third view, like what is done by the ‘Reformat widget’. The idea is to rotate a single view quickly, without need for the reformat widget, which requires zooming and positioning.
Of course, the Reformat widget does the job already.
