In a project we might need to mount the Slicer installation tree in a read only network share, with the assumption that all user configs and extensions can be installed under user’s own folder, perhaps something like /home/murat/.slicer/, including the ini files. We only need this to work on Linux.
Is this possible?
While the extension path is configurable and indeed I can set it up to a place like /home/murat/.config/slicer.org/Extensions from preferences, Extension manager seems to want the slicer.org/Slicer-XXXXX.ini under the installation tree to be writable.
Is there an alternative?
Also what about the user/extension install python libraries? I didn’t see a configuration for that. Do all go to the site-packages folder under the installation tree?
It’s not set up for that. Better to either pre-install all extensions in the read-only directory or give each user their own copy of Slicer to install extensions in.
Understood. Is this somewhere on the roadmap?
It would facilitate easier deployment of Slicer on centrally managed institutional servers and HPC type systems (all Linux based).
I remember we had looked into it at one point and the sticking issue as I recall is the python package folders, where different extensions can install different packages. Probably it’s possible to do something like symlink all the other files and directories to the readonly location and leave some as real directories in the user’s space, but that sounds kind of fragile. If people are using HPC resources then a couple gig of space for their software doesn’t seem to much to ask, especially for just this one use case where they need user-specific extensions.
Thanks. We will keep in mind.
The issue is not about the space. It is about management and update. Centrally managed, closed resource like HPC systems may not even give you as a user external internet access…
Well, in that case you wouldn’t be able to install extensions anyway so installing a read-only shared version of Slicer shouldn’t be a problem.
You can always send --additional-module-paths on the command line to add modules, you just wouldn’t install python packages the normal way. You could just put them in a place where your custom module knows where import them from (e.g. by adding a path sys.path)
The user-specific config is already in ~/.config/slicer.org/Slicer.ini. What else were you concerned about?
We are trying to figure out what work, what creates other issues. There is a potential to deploy Slicer in the our central HPC system.
If we overcome the internet access issue, and let people install Slicer into their home directories, the issue is not the space. There will be a bunch of stale slicer installation. We are not talking about people who would be using Slicer day and out and necessarily be aware they are using a version from two years ago.
I personally preferred a solution, where people do something like
module load Slicer,
which brings the latest stable (HPC admins responsibility)
and then the first time they use it they bookmark their extensions (which are saved in a place ~.config/slicer.org/Extensions-XXXXX, or ~./config/slicer.org/python-packages, etc). So they will always be using whatever the latest stable version system provides, and if this upgrades than all extensions and packages are automatically repopulated (or that’s roughly the idea).
Anyways, we will do some experiments and report…
Is the use of an HOC admin a solution to a more general issue of people not knowing there is a new stable release available and their decision to not update to it? Should there be a focus to improve the Slicer update process to encourage more people to use the latest stable? Would this avoid the need for such a central read-only Slicer solution?
Not just that, admin is required to make Slicer work with their self-signed certificates and network settings. it makes much more sense this is done once for everyone (and checked for every update).
I think we are currently doing a OK job of notifying the stable updates in the software (it might be a bit more prominent then just a text in welcome page). And extension update mechanism works great.
For a similar situation where most users want to use a read-only version of slicer maintained by the admins, we added a slicer directory to everyone’s path that had scripts to launch specific versions with any customizations needed for that system.
E.g. there was a script called slicer522 that did this:
#!/usr/bin/env bash
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
LD_LIBRARY_PATH=${SCRIPT_DIR}/openssl:${SCRIPT_DIR}/krb5/src/lib:${LD_LIBRARY_PATH} ${SCRIPT_DIR}/Slicer $*
You could do something similar to point to read-only versions with SlicerMorph installed.
For users like you describe it’s important that they use specific versions so they have consistent behavior - changing the program that appears when they type slicer would add confusion and possibly change their experimental results.
For users who need to install specific extensions they should just be able to copy your customized version into a writable space.
I think you want to minimize the amount of custom development work to support a particular HPC system. You want users to develop skills that will transfer to other situations.
I’ve been facing a similar issue with the Slicer-Flatpak project. I ended up having a couple of patches to enable the extensions and pip-installed python packages land in local folders (i.e., /home/rafael/.local…); as far as I remember the patches were working fine. @muratmaga If your use-case could be based on a custom-built Slicer, this could be an option and I can help you set it up.
During last project week I was discussing with @jcfr on the idea of enabling this behavior in runtime (launcher flag?), but we didn’t concluded on any solution. Something like this could be useful in these scenarios and to wrap the Slicer binaries in a Flatpak, instead of building a custom Slicer. @pieper, @lassoan, @jcfr could something like this make sense?
What happens when an extension installs a Python package that is already bundled with Slicer (but a different version)? Does the Python package installed in the user folder override the Python package bundled with Slicer core?
If yes then we can add this patch to Slicer core and make it easier to activate this option (right now I think it can only be enabled with a CMake flag at build time). The launcher needs to know about this, so adding a flag to the SlicerLauncherSettings.ini file is a good place to store the flag.
I would have to check for a definitive answer on this, but if I remember correctly, the one installed on the user’s folder will take precedence. We might be able to change this behavior, though.
Enabling something like this would enable us to wrap the official Slicer binaries on flatpak and have one more distribution channel. My current implementation of the Slicer-Flatpak is a custom build and still have some ABI compatibility issues with extensions.
I did not consider the custom build. The issue with custom build we will need to build all the extensions as well. We will discuss these options see which one is going to require the least administrative oversight.
But I think it will be nice to have a solution that allow centrally managed Slicer installation, with options for users to customize.
M
That’s usually handled by modules module. You can maintain independent versions of the sofware. Whichever one you activate provides the executable you wish.
So, module load slicer may bring the latest stable but you can conitnue to have the previous one through module load slicer-5.4.1 (or whatever you want to call). That’s usually how multiple versions of the same software package is maintained in hpc systems.
I second this. I came to this tread as I automated installation of Slicer using Easybuild (to have module load Slicer ), but quickly realized, that there is no good mechanism how to deal with user-managed extensions with centralized Slicer install.
I was able to create proof-of-concept based on fuse-overlays.
TL/DR: we create per-user overlay fs on top of read-only baseline install of Slicer.
Proof of concept Setup:
(as sw repository admin: - steps with a. prefix)
a.1) Install slicer into shared, read-only directory (in our case Cern VM fs)
a.2) apply all baseline workarounds (eg. delete libcrypto.so.* )
result: We have working Slicer in /cvmfs/sys.sw.nudz/software/Slicer/5.6.1_bin, but we’re not able install extensions.
(as system admin - steps with b. prefix)
b.1) make sure that fuse-overlayfs is available at compute nodes
proof of concept:
(as user at compute node steps with c. prefix)
prepare directories and export them as environment variables:
c.1.1): create directory for temporary working data of overlayfs and export it as WORKDIR
(assuming we’re inside of Slurm job, so using SLURM_JOB_ID env. variable )
export WORKDIR="${HOME}/.cache/slicerfs/${SLURM_JOB_ID}"
mkdir -p $WORKDIR
c.1.2): create directory for per-user persistent data and export it as UPPERDIR
TODO: we should make this directory unique for every Slicer installation
export UPPERDIR="${HOME}/.local/slicerFS"
mkdir -p $UPPERDIR
c.1.3): create mountpoint for overlayfs in user home.
mkdir ${HOME}/SlicerMount
c.2.1): Mount overlayfs
fuse-overlayfs -o lowerdir=/cvmfs/sys.sw.nudz/software/Slicer/5.6.1_bin,upperdir=${UPPERDIR},workdir=${WORKDIR},squash_to_uid=${UID} ${HOME}/SlicerMount
now we see the mount in df:
user@sup204:~$ df | grep Slicer
fuse-overlayfs 15300820992 13382676352 1918144640 88% /home/user/SlicerMount
user@sup204:~$
c.2.2): we can run Slicer by typing
SlicerMount/Slicer
possibly install some extensions, restart app, and see extensions files here:
user@sup204:~$ ls -1 ${HOME}/.local/slicerFS/slicer.org/Extensions-32438/
ABLTemporalBoneSegmentation
ABLTemporalBoneSegmentation-icon.png
ABLTemporalBoneSegmentation.s4ext
SlicerElastix
SlicerElastix-icon.png
SlicerElastix.s4ext
user@sup204:~$
c.3.1): Umounting overlayfs
fusermount --unmount ./SlicerMount/
Conclusion
we were able to workaround issues coming from concepts used in Slicer. Follow-up tasks are to investigate feasibility of implementation steps indicated in this P-O-C using current state-of-the-art module system, Lmod.
Would this work for python packages installed through pip_install inside Slicer?
Thanks for sharing all these details @jose-d.
I am wondering if it is OK to install large Python packages in user-writable folders (e.g., Pytorch size is 5-10 GB when unpacked) then could the Slicer package (less than a GB when unpacked) be installed in the same user-writable folder? The Slicer executable would then be just a script that installs Slicer (downloads the package, unpacks it, applies any patches), and run the application. If Slicer is already installed then it would just run it.
That said, if overriding preinstalled Python packages by ones installed in user folders work then that is worth exploring, because then commonly used large packages could be preinstalled, saving some storage space.