Running Slicer on headless server (Installing Extensions and more...)

[Davide_Cester]

Dear all,

I am setting up a shared server for the research in our group, it’s a headless Ubuntu 20.04. People are supposed to log in to web services and share the computing power. JupyterHub is already up & running, with Python and R kernels available. Now I would like to add Slicer support, which I already did on a standalone machine with Ubuntu 18.04.

The documentation (Documentation/Nightly/Developers/Python scripting - Slicer Wiki) refers to a discussion that is not reachable ([slicer-devel] Slicer batch mode), that’s why I am opening a new one here.

I have installed Slicer in /opt/slicer-4.10.2 and I am trying to adapt the installation instructions in step 2 from here: GitHub - Slicer/SlicerJupyter: Extension for 3D Slicer that allows the application to be used from Jupyter notebook

My questions are:

• I tried to run the Slicer GUI from remote with “ssh -X” but it loops erroring “composeAndFlush: makeCurrent() failed” after the initial medical disclaimer. Is that supposed to work?
• how do I install the SlicerJupyter extension without the GUI?
• how can I access the command to be copied in the SlicerJupyter Extension tab?

Thanks in advance

[Davide_Cester]

Here some of the solutions I found so far:

@Davide_Cester

how do I install the SlicerJupyter extension without the GUI?

https://www.slicer.org/wiki/Documentation/Nightly/Developers/FAQ/Extensions#How_to_manually_download_an_extension_package.3F
However the site won’t load normally, see Extension app store not accessible (http://slicer.kitware.com/midas3/slicerappstore) for a workaround. Even in this case no extension is found, for any combination of OS and version…?
That site seems fundamentally broken, everything I clic just generates an endless loading in the browser.

@Davide_Cester

how can I access the command to be copied in the SlicerJupyter Extension tab?

From https://github.com/Slicer/SlicerJupyter:
jupyter-kernelspec install /tmp/SlicerJupyter-build/inner-build/share/Slicer-4.11/qt-loadable-modules/JupyterKernel/Slicer-4.9/ --replace --user
Of course the paths will have to be updated but that depends on the extension installation, which I’m not yet able to do…

[lassoan (Andras Lasso)]

@Davide_Cester

However the site won’t load normally, see Extension app store not accessible

I confirm that the extension server takes really long time to respond (it took me now about 5-10 minutes to load the page). Unfortunately, the server is temporarily overloaded from time to time. It usually returns to normal in an hour or so. @jcfr @Sam_Horvath when you have a chance, could you check if anything can be done?

In the meantime, you can try specifying a Slicer revision - then the server can skip the automatic revision detection step and there is a higher chance your request gets served. For example: @KitwareMedical/slicer-extensions-webapp

If that does not work then you can download extensions directly from the dashboard: CDash

@Davide_Cester

I have installed Slicer in /opt/slicer-4.10.2 and I am trying to adapt the installation instructions in step 2 from here: https://github.com/Slicer/SlicerJupyter

This docker image describes a complete Slicer Jupyter kernel installation:

We use this image with binder’s dockerhub and it works well.

[Davide_Cester]

Hi,

thanks for the reply! It now works and I’m writing a few lines as reference for the others.

After some minutes I managed to get the archive for the extension, and I figured it out where to manually copy the files. In the install.sh in the Docker repo there is a nice command-line alternative which I didn’t fully test: one command to get the current version, another to download the extension, the third one to install it. However with my slicer the first one fails with something like ‘revision’ property missing, so I did it manually.

To run headless I installed the Ubuntu package xvfb. In the docker repo a custom Xorg is launched instead, with xorg.conf and all, but I am confortable with xvfb from previous experiencese. I located the kernel-template.json, copied into kernel.json in the same folder and replaced the version and executable parameters:

{
"display_name": "Slicer 4.10",
"language" : "python",
"argv": [
    "xvfb-run",
    "-a",
    "/opt/slicer-4.10.2/Slicer",
    "--no-splash",
    "--python-code",
    "connection_file=r'{connection_file}'; print('Jupyter connection file: ['+connection_file+']'); slicer.modules.jupyterkernel.startKernel(connection_file);slicer.util.mainWindow().showMinimized()"
]

}

and finally I installed the kernel in Jupyter with

sudo jupyter-kernelspec install /path-to-slicer/share/Slicer-4.10/qt-loadable-modules/JupyterKernel/Slicer-4.10/ --replace

Didn’t add the documented option --user to enable the kernel for the entire system.

I have now a nice server where user can use Slicer via JupyterHub

[lassoan (Andras Lasso)]

I’m happy that it works for you now.

I would strongly recommend to use Slicer-4.11:

• it uses Python 3
• you can install any python packages into Slicer’s Python environment
• its kernel is based on xeus-python which is essentially a full IPython implementation (while our custom kernel that we implemented in Slicer-4.10 had a number of limitations), including widgets and interactive 3D viewer in notebook cells. See some more information here:

Run Slicer in your web browser - as a Jupyter notebook or as a full application

It has been possible to use Slicer from Jupyter notebooks but the most recent Slicer Preview Release takes this to a whole new level. Click here to try a few example interactive notebooks now - or see a short video demonstrating the new features: [Medical image processing in your web browser using Jupyter notebooks and 3D Slicer] Highlights: Improved interactive use You can now use IPython widgets (sliders, buttons, etc.) to control Slicer or adjust processing and visualization…

[Davide_Cester]

I was about to ask, why I was not seeing the JupyterNotebooksLib module. And now I wonder why I installed 4.10…??? I’ll install 4.11 while I’m “hot”…
I got the Preview announcement via mail a few days ago, that’s what motivated me to integrate Slicer in the server I’m building. Very nice tool!

Thanks again!

[lassoan (Andras Lasso)]

Is your Slicer/Python/R JupyterHub docker image publicly available? It would help people who wants to set up Slicer server on their computers.

[Davide_Cester]

Unfortunately it’s not an image, I’m not that much into Docker yet. It’s an “old style” server with several services.
I’m planning to write an article as soon as my first two users provide positive feedback, which I hope will be soon. Later I could maybe do an image myself, or someone could use my notes to build it. It would definitely be a good idea.

[Davide_Cester]

Ok, I moved to Slicer 4.11. Same procedure as before: installed slicer, installed extension, installed kernel. Kernel is available in Jupyter, but hangs in the connection and never becomes operational.

Usually when a kernel is hanging in the init / conn stage it’s a websocket issue, I had it before. But here the Python kernels are working, and Slicer 4.10 was working with the same webserver configuration. If I manually run the headless slicer it doesn’t crash, so it looks really like a communication problem.

Is there anything different between the two versions of Slicer and the extension?

Something I’ve noticed is that v. 4.11 has a kernel-configure.py script. Is that important? How do I run it?
Here’s my kernel.json:

{
    "display_name": "Slicer 4.11",
    "language" : "python",
    "argv": [
        "xvfb-run",
        "-a",
        "/opt/slicer/Slicer",
        "--no-splash",
        "--python-code",
        "connection_file=r'{connection_file}'; print('Jupyter connection file: ['+connection_file+']'); slicer.modules.jupyterkernel.startKernel(connection_file);slicer.util.mainWindow().showMinimized()"
    ]
}

My revision number is now 29074.

EDIT: it’s definitely a websocket problem, I get Javascript errors on the browser’s console.

EDIT #2: I have re-installed Slicer 4.10 in parallel, now I see both kernels. 4.10 works, 4.11 has WS issues.

[lassoan (Andras Lasso)]

@Davide_Cester

Something I’ve noticed is that v. 4.11 has a kernel-configure.py script

Where have you found that script? Can you send a link to the repository/file?

@Davide_Cester

I have re-installed Slicer 4.10 in parallel, now I see both kernels. 4.10 works, 4.11 has WS issues.

Check all configuration steps that are performed in the slicer-notebook image - in the dockerfile and in install.sh.

Also double-check your zeromq version in Slicer and in the Jupyter client (if they are matching or compatible; maybe try upgrading/downgrading to a few different versions).

[Davide_Cester]

@lassoan

Where have you found that script? Can you send a link to the repository/file?

It’s inside 29074-linux-amd64-SlicerJupyter-git4c5fa4d-2020-05-23.tar.gz downloaded from midas, in the folder 29074-linux-amd64-SlicerJupyter-git4c5fa4d-2020-05-23/share/Slicer-4.11/qt-loadable-modules/JupyterKernel/Slicer-4.11

@lassoan

Check all configuration steps that are performed

You are right… I think what I was missing was:

sudo /opt/jupyterhub/bin/python3 -m pip install ipywidgets ipyevents ipycanvas
sudo /opt/jupyterhub/bin/python3 -m pip install websockify --upgrade
sudo /opt/jupyterhub/bin/jupyter-labextension install @jupyter-widgets/jupyterlab-manager
sudo /opt/jupyterhub/bin/jupyter-labextension install @jupyter-widgets/jupyterlab-manager ipycanvas
sudo /opt/jupyterhub/bin/jupyter-labextension install @jupyter-widgets/jupyterlab-manager ipyevents

commands to be run in the environment of jupyterhub.

Thanks!!!

[jcfr (Jean Christophe Fillion Robin (Kitware))]

@lassoan

Unfortunately, the server is temporarily overloaded from time to time

Indeed, the server is currently on “life support” (we need to allocate resources to transition it), in the meantime I will make sure it is restarted on daily basis (9pm EST)