I’d like to set up a CI-pipeline with Gitlab CI, which should enable continuous testing of our extension (in python). However, it’s not quite obvious to me which dependencies I therefore have to integrate. The imports i need from Slicer are basically slicer and qt.
To fasten up the process, I’ve used the docker-image slicer/slicer-build mentioned in this thread and described on github.
However, I still have the following questions:
Is it correct that these images are the official slicer docker-images?
Is it correct that they represent the lastest nightly build?
What is the path I have to add to PYTHONPATH, in order to import these distributions? (I’d have guessed /usr/src/Slicer-build/Slicer-build/bin/Python but I’m not quite sure…)
Thank you for any help or suggestions in advance!
Once I have everything in place, I’d like to share it so that others can profit as well.
The slicer/slicer-base image is published daily and only contain the dependencies of Slicer in the form of build trees (VTK, ITK, …). It is used to make sure the changes proposed as Slicer pull request can compile. This image depends on slicer/buildenv-qt5-centos7
I’ve now built a docker image based on slicer/slicer-build, to which I copy my extension into (also via a container).
Using the cmake command from SlicerITKUltrasound, I managed to build my extension.
I think the only thing I now need is to set up a CTestTestfile.cmake for my python tests, that I have already written and which run trough locally.
Do you agree with my thought process?
Btw. my Dockerfile currently looks like this:
FROM alpine:latest as base
RUN apk add git
RUN git clone my_extension
FROM slicer/slicer-build as build
RUN cd /usr/src && mkdir my_extension-build
COPY --from=base /my_extension /usr/src/my_extension
RUN cd /usr/src/ && mkdir my_extension-build && cmake -DSlicer_DIR:PATH=/usr/src/Slicer-build/Slicer-build -DBUILDNAME:STRING=my_extension /usr/src/my_extension
RUN cd MyExtension && make
I cannot comment on that if it is correct or optimal (@jcfr should give feedback on that), but we should probably use this in our extensions and also add it to our extension template. @jcfr@pieper what do you think?
I don’t think there should be anything special about running in the container - if you build the extension structure using the wizard then all the build and test infrastructure should be set up automatically. But yes, if there’s a way to build containerized testing infrastructure into the extension template we should.
@unnmdnwb3 - do you really want to do the cmake/make commands in the RUN statement? I’d think you’d want this container to be doing the checkout and build in the instance and not in the docker build step.
We could add all the necessary files to the extension template (docker subfolder, travis & circleci configuration files, badges in the readme.md file, etc.). It should be easy if there is a good example to follow.
That’s a great idea! Everything that automates testing is awesome. Facilitating the CI could support teams to start using it, which supposedly results in better tested and hence more robust extensions.
I’d be happy to later contribute by Dockerfile and gitlab-ci.yml once everything’s in place - if you’re interested of course. And I could also propose a .travis.yml for those working with Github.
Thanks for pointing that out. I honestly don’t have a rich experience with cmake and joined the extension project a few days ago. If you advice me to do it differently, I’ll follow your suggestion!
I’ve now managed to get a correct build (as described in the wiki) and configuration in my CMakeLists.txt, so that the correct tests are run.
But now I have the following issue:
3: Test command: /usr/src/Slicer-build/Slicer-build/Slicer "--no-splash" "--testing" "--launcher-additional-settings" "/usr/src/extension-build/AdditionalLauncherSettings.ini" "--additional-module-paths" "/usr/src/extension-build/lib/Slicer-4.9/qt-scripted-modules" "/usr/src/extension-build/lib/Slicer-4.9/cli-modules" "/usr/src/extension-build/lib/Slicer-4.9/qt-loadable-modules" "--python-code" "import slicer.testing; slicer.testing.runUnitTest(['/usr/src/extension-build/Extension', '/usr/src/extension/Extension'], 'extensionTest')"
3: Test timeout computed to be: 1500
3: SlicerApp-real: cannot connect to X server
3: vtkDebugLeaks has found no leaks.
3/3 Test #3: py_extensionTest ...................................***Failed 0.38 sec
@unnmdnwb3, looks close, but you need to have X up running for slicer to run.
If you look at the X11 example in the SlicerDockers repository you can see how to set this up. Also in the readme there is an example of running a script inside the instance that uses Slicer and the X server. This approach could be adapted to run the testing as well, probably using the build images.
There’s clearly a lot of potential benefit here, but things aren’t really documented or coordinated. It’ll be good to see a good clean end to end solution.
Short update: I’ve now managed to (mostly) resolve the X issues.
When running ctest -V though, the following error occurs:
2: Test command: /usr/src/Slicer-build/Slicer-build/Slicer "--no-splash" "--testing" "--launcher-additional-settings" "/usr/src/extension-build/AdditionalLauncherSettings.ini" "--additional-module-paths" "/usr/src/extension-build/lib/Slicer-4.9/qt-scripted-modules" "/usr/src/extension-build/lib/Slicer-4.9/cli-modules" "/usr/src/extension-build/lib/Slicer-4.9/qt-loadable-modules" "--python-code" "import slicer.testing; slicer.testing.runUnitTest(['/usr/src/extension-build/Extension/Logic', '/usr/src/extension/Extension/Logic'], 'DExtensionTest')"
2: Test timeout computed to be: 1500
Could not init font path element unix/:7100, removing from list!
2: Number of registered modules: 101
2: Traceback (most recent call last):
2: File "<string>", line 1, in <module>
2: File "/usr/src/extension-build/lib/Slicer-4.9/qt-scripted-modules/Extension.py", line 15, in <module>
2: from pathlib import Path
2: ImportError: No module named pathlib
2: loadSourceAsModule - Failed to load file "/usr/src/extension-build/lib/Slicer-4.9/qt-scripted-modules/Extension.py" as module "Extension" !
2: Fail to instantiate module "Extension"
2: Number of instantiated modules: 100
2: Number of loaded modules: 100
2: Switch to module: "Welcome"
2: -------------------------------------------
2: path: ['/usr/src/extension-build/Extension/Logic', '/usr/src/extension/Extension/Logic']
2: testname: ExtensionTest
2: -------------------------------------------
2: Traceback (most recent call last):
2: File "<string>", line 1, in <module>
2: File "/usr/src/Slicer-build/Slicer-build/bin/Python/slicer/testing.py", line 22, in runUnitTest
2: suite = unittest.TestLoader().loadTestsFromName(testname)
2: File "/usr/src/Slicer-build/python-install/lib/python2.7/unittest/loader.py", line 91, in loadTestsFromName
2: module = __import__('.'.join(parts_copy))
2: File "/usr/src/extension/Extension/Logic/ExtensionTest.py", line 3, in <module>
2: from .ExtensionLogic import *
2: ValueError: Attempted relative import in non-package
2: Switch to module: ""
error opening security policy file /usr/lib64/xserver/SecurityPolicy
2: vtkDebugLeaks has found no leaks.
It seems like a relative import is not possible. However the Slicer Python should be 3.+, right?
Possibly, this is because Slicer is not correctly integrated into the project and uses the default installation?
And is it correct, that slicer/slicer-base pulls from the master and hence ends up with python2?
I tried to find PythonSlicer (and found it locally on my mac), but wasn’t able to locate it on the slicer/slicer-build image. However, I found a SlicerPython which is an executable as well. Can you elaborate on the difference between them?
We renamed “SlicerPython” to “PythonSlicer” , because PyCharm only accepts custom Python interpreter names that start with “python”. We kept “SlicerPython” for backward compatibility for a while, but it will be removed in the future. The two processes do exactly the same.