How could I not know about this Binder 
?? Amazing
Would be glad if you could give a recommendation for a Slicer capable cloud provider if that is possible here … Thank you Andras
@pieper what are your latest recommendations for this?
At this point I would say Amazon Workspaces is the perhaps the easiest way to get a cloud installation of Slicer, although it is still very enterprise-focused, somewhat flakey, and may be expensive for routine use. It provides a standard windows machine where you install the regular Slicer app and extensions just like you would on a local machine. If you just boot it up when you need it (it takes several minutes longer than it should) and then shut it down when you are done it’s a very cost effective way to do things. You still need a reasonably fast network but the interactivity is pretty good in my experience. Amazon also has something called App Streaming, which is also windows, but without the usual desktop features, so it’s not clear how practical that will be for getting real work done.
Depending on your needs there are also custom virtual machine and container images that are described here. Some would be better for batch processing or faster startup times. GPU drivers in the linux VMs is a constant headache as are boot times. Sometimes you can get a docker instance to boot within a minute, which makes it more practical for on-demand use. On my local linux machine I can boot a docker instance of slicer almost as fast as a native executable (under 10 seconds) but the cloud providers are always slower because it seems they allocate a VM first to run the container.
For the SlicerMorph workshop @muratmaga is running about 15 docker instances per-VM on Azure so that users effectively timeshare CPU and memory. I’m curious to hear how that approach works out in practice.
So cloud usage of Slicer is very promising, but there are practical issues with the current options.
2 Likes
I will let you know next week 
I should say also that if you are comfortable with the cloud computing consoles you can also boot Windows VMs. They come with server/datacenter versions of windows, sometimes very locked down by default, but with the right configuration they can feel just like a PC. In the past I’ve been able to get Slicer and even a whole Slicer development environment installed to build from source. But I’m biased towards linux in spite of the gpu driver issues.
We just finished the first day running SlicerMorph workshop on the Cloud. This note is for sharing the experience with other folks interested in running Slicer as an remote interactive desktop application (not thru Jupyter notebooks or binder).
-
We are using this docker GitHub - SlicerMorph/SlicerMorphCloud: Docker for running SlicerMorph on Cloud, which provides a simple openbox WM, with a file browser, Slicer, R/Rstudio and Firefox. (same docker can be used for GPU accelerated setups with Nvidia cards too). Docker image is huge though (about 25GB).
-
For every 10-12 students, we have spun a HB120rs_v2 node on Azure (480GB of RAM, 120 non-threaded cores. No GPU. About $3.5/h). Each node also have a 1.2TB persistent storage as their work area and workshops data.
-
We spun individual containers for every attendee. Every user gets a one-time use password and a unique URL (both of which unfortunately changes if have to restart their instance).
-
They use Firefox installed inside the docker to bring data in (e.g., from MorphoSource or from their dropbox account).
-
In my experience, dedicated TurboVNC clients gives a much more interactive experience then noVNC running in a web browser (both in terms of bandwidth usage and image quality).
-
User experience is highly dependent on their bandwidth and connection quality (for example for me it is as good as being in the lab that use LAN settings). We could have improved this a bit more by spinning nodes in other geographical areas and cluster the people accordingly. Next round we might do that. And if users are beyond firewall, ephemeral tcp ports we use sometimes are blocked and unfortunately that’s not something we can do about (apart from finding a better solution for having distinct accounts for everyone).
Today, out of ~40 users only 1-2 needed their docker instance restarted (which is mostly due to a design issue where if you exit out of the WM session by choosing Exit from window manager’s popup menu, it kills the VNC session. We need to remove that menu). And I ran my entire tutorial (~4h) using only the cloud instance.
But right now usage is lightweight. Tomorrow is segmentation and we will see if we encounter any more issues.
Overall, we managed to cover the materials much faster, because we didn’t get bogged down strange OS specific issues (locales, folders with non-ascii characters, desktops hijacked by cloud file share, missing libraries) that we had to troubleshoot over zoom connection. To be clear, these issues continue to happen on desktops (as we encourage them to have a local computer as a backup) but didn’t take it away from workshop time.
Overall, the experience can be improved with more careful planning (this was our first attempt, so we are also learnign). But Slicer on the cloud as an interactive application, is a very real thing.
| CONTAINER | NAME | CPU | % | MEM | USAGE | / | LIMIT | MEM | % | NET | I/O | BLOCK |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 04fe810bc45c | 0.01% | 1.162GiB | / | 100GiB | 1.16% | 423MB | / | 3.08GB | 778MB | / | 401MB | 118 |
| 7367b651ded8 | 0.07% | 5.619GiB | / | 100GiB | 5.62% | 3.08GB | / | 4.26GB | 17.3MB | / | 309MB | 727 |
| 9a29d3cc4f67 | 0.00% | 56.1MiB | / | 100GiB | 0.05% | 60.8kB | / | 691kB | 41kB | / | 0B | 55 |
| 3bb70ca00cb0 | 0.05% | 1.025GiB | / | 100GiB | 1.02% | 75.7MB | / | 572MB | 222MB | / | 67.8MB | 130 |
| 850f145587ea | 0.09% | 886.3MiB | / | 100GiB | 0.87% | 443MB | / | 806MB | 10.8MB | / | 416MB | 302 |
| 545cc4fcb27c | 0.09% | 1.004GiB | / | 100GiB | 1.00% | 36.8MB | / | 42.8MB | 11.5MB | / | 86.8MB | 236 |
| 307d4b87f8ea | 0.05% | 3.378GiB | / | 100GiB | 3.38% | 1.5GB | / | 2.18GB | 239MB | / | 493MB | 546 |
| 065a1b06903c | 5.33% | 1.902GiB | / | 100GiB | 1.90% | 1.97GB | / | 2.65GB | 17.1MB | / | 508MB | 649 |
| 8e674178b5d7 | 0.06% | 3.557GiB | / | 100GiB | 3.56% | 544MB | / | 3.46GB | 12.7MB | / | 289MB | 361 |
| 234ae464a44f | 0.05% | 1.376GiB | / | 100GiB | 1.38% | 3.25GB | / | 978MB | 4.72GB | / | 303MB | 275 |
| ccd3167e3bbd | 2.32% | 1.489GiB | / | 100GiB | 1.49% | 482MB | / | 1.48GB | 24.3MB | / | 174MB | 512 |
| baa4337b35ca | 0.05% | 1.321GiB | / | 100GiB | 1.32% | 425MB | / | 1.33GB | 568MB | / | 343MB | 278 |
5 Likes
Very useful to know - thanks for the detailed report and glad you know the workshop went so well.
Hi. First of all: thank you very much to the creators of GitHub - SlicerMorph/SlicerMorphCloud: Docker for running SlicerMorph on Cloud
I corrected the download links of the dockerfile and edited the nogpu.sh file here:
So now it builds successfully.
Following the instructions of README.md I created a container for one user executing:
for user in $(cat users); do ./nogpu.sh $user ; done > instance-table.csv
When I access the link given to me I get the following:
Can you help understand what is happening? Do I need to edit these lines?
I also created an ubuntu installation on my hard-disk and executed each one of the docker commands. Then I try launching a vncserver and access the corresponding link and it yields the same result “File Not Found”. So I suspect maybe there is some configuration that could be corrected.
First comment, that’s a bit outdated and a was a very early attempt, and not particularly flexible to use.
Is accessing through a web browser a must? When we did the workshop, I did not like the performance of noVNC, which used 2-5 times more bandwidth than turbovnc, and was less interactive. So now are using TurboVNC dedicated client, which interacts nicely with the session manager of TurboVNC server and handles some of the complexities of starting the individual sessions. But this means your users need to install Turbovnc software.
In the script above, users file should contain the usernames of your cloud uses, and instance table ends up containing the specific URL (hostname + port) and the one time password associated with authentication.
I again, I wouldn’t do (and we are not doing) this way now. TurboVNC session manager is handling all the authentication via SSH. I also stopped putting the Slicer inside the container, for two reasons:
- Updating slicer requires rebuilding the container
- You will need to map persistent storage for your users to keep their data between sessions anyways (unless it is a short course or something).
So I simply copy the fully functional Slicer folder to their persistent storage, which also gives them the flexibility of installing additional extensions (not have to do this everytime their docker instance starts).
take a look at this branch github.com/SlicerMorph/SlicerMorphCloud/tree/eglbackend
This is what we are using now. Couple things to consider. We want the ability the to be able to size the user’s session to match their data (perhaps they need GPU, perhaps they need large RAM etc), and also provide analytics platform (mainly R), so there is probably a lot of unnecessary clutter for you.
You should probably delete all R related lines in the dockerfile to cut down your build time (unless you need it of course). I directly download the Slicer image onto the host and unpack to the users persistent storage .
Hi Murat. Thanks for the answer. We are not bandwidth-constrained and we would prefer to use a web browser.
I made these few changes to your main branch in my fork: Successful build docker image by mauigna06 · Pull Request #1 · mauigna06/SlicerMorphCloud · GitHub
I know the main branch worked well for you in the classes you gave. Do you have any idea why the web-browser shows “File Not Found” considering I followed the instructions step-by-step?
Not sure, are you running the docker and the client on the same box? Is that really the hostname? Did you try replacing the with localhost (if it is local). We would manually change the first to match the IP address of the box to connect.
I’m running the docker container and the web-browser on the same computer.
Is that really the hostname?
Yes, that’s the hostname: it is what appears first on my terminal after “root@”
We would manually change the first to match the IP address of the box to connect.
I changed the link to: http://127.0.0.1:49156/vnc.html?host=127.0.0.1&port=49155&resize=remote and the result is “File Not Found”
But if I stop the container the same link shows this:
It says “Cannot connect” on the heading and “Try again” in the blue button.
So I think TurboVNC is working but needs some extra configuration because when the container was live it showed the “File Not Found” page and the small VNC icon on the top of the firefox’s tab.
I suspect you are correct. But it is hard to debug, because the entire VNC session lives in the docker container. I guess you can jump into the running docker container and take a look at the log file under /home/docker/.vnc to see what file is missing, or what the actual error is.
That was one of the reasons why we moved away from this.
1 Like