Plot images from PNG files in slicer jupyter kernel

[BennetMontgomery]

I have a Jupyter Notebook that uses a Slicer kernel and I would like to plot a variable number of captures of the 3D view in the notebook, each view captured from different scenes loaded sequentially. The way I have it set up now is the notebook takes shots of the 3D view and writes them to png files, and I would like to load the png files into the notebook and plot them in a table.

I’ve tried matplotlib and ipyplot, but they don’t work with a slicer kernel in jupyter notebooks. The most potentially useful resource I’ve found is the Slicer Jupyter library, but I don’t see any way to use it to load and plot png files. Is anyone aware of a way to plot png files in jupyter notebooks using a slicer kernel?

[lassoan (Andras Lasso)]

@BennetMontgomery

The way I have it set up now is the notebook takes shots of the 3D view and writes them to png files, and I would like to load the png files into the notebook and plot them in a table.

This is already implemented with a one-liner for slice views:

By changing a few lines in ViewLightboxDisplay script (essentially, replace screenCaptureLogic.captureSliceSweep by screenCaptureLogic.capture3dViewRotation, you can create a table showing 3D view in different orientations. I’ve created this using Screen Capture module GUI, you can play a bit with the GUI, too to get to know a bit better what the module can do:

Finally, here is an example of a script creating some data augmentation for training data and generation of an overview image of all data sets:

Generating augmented training data from nrrd file using random translations, rotations, and deformations

I’ve updated the volume augmentation script to apply random translations and rotations in addition to the random deformations, and to also take screenshots so that you can get a quick overview of how the augmented data looks like. Gallery of example output: [image] Script that generated it:

[BennetMontgomery]

Thanks for the help but unfortunately, this doesn’t solve my problem. Let me just clarify what I’m trying to accomplish:

I’m loading mrb scenes into the jupyter managed slicer, constructing a volume, rendering the volume, and then capturing a screenshot of the 3D view for display. The number of scenes is variable and I would like for the user of the notebook to load as many scenes as they like. At present, I have a cell that allows the user to input paths to mrb scenes to load into the notebook:

Later on, I use a for loop to load one of the scenes, do the processing, capture the 3d view, and repeat until all scenes have been processed:

In that cell I try using the lightbox display, but neither it nor the view3ddisplay() method seem to work when embedded in for loops as both give an identical result:

Everything works fine but no images are displayed in the notebook. The view3ddisplay() method works when not embedded in a for loop but I don’t see a way to retain a theoretically limitless number of scenes to test on without it. As I see it, my only option is to get rid of the for loop and abandon being able to load a limitless number of scenes but I would really like to avoid that if at all possible.

Thanks!

[lassoan (Andras Lasso)]

Jupyter notebook shows only the last result. If you want to show multiple results then you need to put each result that you want to display as input argument of display() function.

For example, if you want to display 3 slice views in a for loop:

This only displays the last one:

for layout in ["OneUpRedSlice", "OneUpYellowSlice", "OneUpGreenSlice"]:
    slicernb.ViewDisplay(layout) # wrong!

This shows all 3 views:

for layout in ["OneUpRedSlice", "OneUpYellowSlice", "OneUpGreenSlice"]:
    display(slicernb.ViewDisplay(layout))

[BennetMontgomery]

Thanks, that works perfectly!