Required extension not being installed in tests

Alex_Vergara · December 9, 2019, 11:10am

While building my extension I have set up a required extension (SlicerElastix) which runs fine, but when I want to test it in Docker it fails with this message:

144 CMake Warning at /usr/src/Slicer/Extensions/CMake/SlicerBlockAdditionalLauncherSettings.cmake:48 (message):
145   Dependent extension SlicerElastix cannot be found by CMake find_package(),
146   therefore paths variables cannot be imported from this extension.  The
147   problem can be resolved by generating SlicerElastixConfig.cmake by adding
148   include(${Slicer_EXTENSION_GENERATE_CONFIG}) to the top-level
149   CMakeLists.txt of the dependent extension.

I have a look at the offending file and indeed, it contains that line. So, I don’t know what it is happening here.

jamesobutler · December 9, 2019, 3:40pm

It appears that another extension, SequenceRegistration, is also having the same issue. That extension depends on Sequences and SlicerElastix. Here’s the nightly Windows build for SequenceRegistration that has the same CMake warning: http://slicer.cdash.org/viewConfigure.php?buildid=1770198.

A previous SlicerElastix error due to being a dependency was brought up in Failing Tests - ImportError: No module named Elastix, but that solution by @jcfr is evidently not working anymore.

lassoan · December 9, 2019, 5:57pm

When you build an extension, you need to pass location of all other extensions that it depends on, as CMake variables. For example, if you want to build Sequence Registration, then you need to set SlicerElastix_DIR=…

For 1-2 extensions with simple dependencies you may do this manually, but it may be simpler (especially when you build several extensions) to use Slicer’s extensions index build script: copy s4ext files that you want to build into a folder and specify that as Slicer_EXTENSION_DESCRIPTION_DIR. See the complete command-line here. This script determines dependencies between extensions, builds them in the correct order and passes all required paths to dependent extensions.

jamesobutler · December 10, 2019, 1:53am

@lassoan Do you have insight on how to fix the factory build of SequenceRegistration extension which specifies SlicerElastix as a dependent extension. Looking to solve this failing test specifically.

lassoan · December 10, 2019, 6:44am

Maybe we just need to add the missing ["Sequences", "Elastix"] dependency declaration here:

github.com

moselhy/SlicerSequenceRegistration/blob/master/SequenceRegistration/SequenceRegistration.py#L20




class SequenceRegistration(ScriptedLoadableModule):

  """Uses ScriptedLoadableModule base class, available at:

  https://github.com/Slicer/Slicer/blob/master/Base/Python/slicer/ScriptedLoadableModule.py

  """



  def __init__(self, parent):

    ScriptedLoadableModule.__init__(self, parent)

    self.parent.title = "Sequence Registration"

    self.parent.categories = ["Sequences"]

    self.parent.dependencies = []

    self.parent.contributors = ["Mohamed Moselhy (Western University), Andras Lasso (PerkLab, Queen's University), and Feng Su (Western University)"]

    self.parent.helpText = """For up-to-date user guides, go to <a href="https://github.com/moselhy/SlicerSequenceRegistration">the official GitHub page</a>

"""

    self.parent.acknowledgementText = """

"""



#

# SequenceRegistrationWidget

#

Alex_Vergara · December 11, 2019, 2:07pm

Adding Elastix in parent dependencies is the detonant for the error

CMake Warning at /usr/src/Slicer/Extensions/CMake/SlicerBlockAdditionalLauncherSettings.cmake:48 (message):
  Dependent extension SlicerElastix cannot be found by CMake find_package(),
  therefore paths variables cannot be imported from this extension.  The
  problem can be resolved by generating SlicerElastixConfig.cmake by adding
  include(${Slicer_EXTENSION_GENERATE_CONFIG}) to the top-level
  CMakeLists.txt of the dependent extension.
Call Stack (most recent call first):
  /usr/src/Slicer-build/Slicer-build/UseSlicer.cmake:279 (include)
  CMakeLists.txt:18 (include)


-- Setting EXTENSION_SOURCE_DIR ................: /builds/project-0
-- Setting EXTENSION_SUPERBUILD_BINARY_DIR .....: /builds/dosimetry4d-build
-- Setting EXTENSION_BUILD_SUBDIRECTORY ........: .
-- Setting MIDAS_PACKAGE_URL ...................: http://slicer.kitware.com/midas3
-- Setting MIDAS_PACKAGE_EMAIL .................: OBFUSCATED
-- Setting MIDAS_PACKAGE_API_KEY ...............: OBFUSCATED
-- Setting EXTENSION_SVNUSERNAME ...............: NOT DEFINED
-- Setting EXTENSION_SVNPASSWORD ...............: NOT DEFINED
-- Setting EXTENSION_DEPENDS ...................: SlicerElastix
-- Setting EXTENSION_BUILD_SUBDIRECTORY ........: .
-- Setting EXTENSION_HOMEPAGE ..................: http://gitlab.com/opendose/opendose4d
-- Setting EXTENSION_CONTRIBUTORS ..............: Alex Vergara Gil (INSERM) and Janick Rueegger (KSA)
-- Setting EXTENSION_CATEGORY ..................: Dosimetry
-- Setting EXTENSION_ICONURL ...................: https://opendose.org/themes/default/images/opendose_favicon.png
-- Setting EXTENSION_DESCRIPTION ...............: This extension contains the entire dosimetry workflow for a multipoint st [...]
-- Setting EXTENSION_SCREENSHOTURLS ............: http://www.example.com/Slicer/Extensions/Dosimetry/Screenshots/1.png
-- Setting EXTENSION_ENABLED ...................: 1
-- Setting EXTENSION_STATUS ....................: NOT DEFINED
-- Setting default for EXTENSION_STATUS ........: 
-- Found Git: /usr/local/bin/git  
-- Found Subversion: /usr/bin/svn (found version "1.7.14") 
-- Configuring Scripted module: Dosimetry4D
CMake Error at /usr/src/Slicer/CMake/SlicerMacroBuildScriptedModule.cmake:78 (message):
  slicerMacroBuildScriptedModule(RESOURCES) given nonexistent file or
  directory 'Resources/JSON/DVK-F-18.json'
Call Stack (most recent call first):
  Dosimetry4D/CMakeLists.txt:53 (slicerMacroBuildScriptedModule)


-- Configuring incomplete, errors occurred!
See also "/builds/dosimetry4d-build/CMakeFiles/CMakeOutput.log".
See also "/builds/dosimetry4d-build/CMakeFiles/CMakeError.log".
e[0KAuthenticating with credentials from /Users/alexvergaragil/.docker/config.json
e[0;me[31;1mERROR: Job failed: exit code 1
e[0;m

lassoan · December 11, 2019, 2:10pm

Make sure to add SlicerElastix_DIR when configuring you project in CMake (or put all s4ext files in a folder and let Slicer extension build script to figure out dependencies, build order, and set all necessary CMake variables).

Alex_Vergara · December 11, 2019, 2:12pm

This is my CI file

image: unnmdnwb3/slicer3d-nightly:0.4.0

build-and-test:
  variables:
    DISPLAY: ":99.0"
  script:
    - mkdir $CI_BUILDS_DIR/dosimetry4d-build && cd $CI_BUILDS_DIR/dosimetry4d-build 
    - mkdir /usr/src/SlicerDICOMDatabase
    - sed  '/\[General\]/a DatabaseDirectory=/usr/src/SlicerDICOMDatabase' /usr/src/Slicer-build/Slicer-build/SlicerLauncherSettings.ini
    - git clone git://github.com/Slicer/ExtensionsIndex.git $CI_BUILDS_DIR/SlicerExtensionsIndex-master
    - cmake -DSlicer_DIR:PATH=/usr/src/Slicer-build/Slicer-build -DBUILDNAME:STRING=dosimetry4d $CI_PROJECT_DIR -DSlicer_EXTENSION_DESCRIPTION_DIR:PATH=$CI_BUILDS_DIR/SlicerExtensionsIndex-master -DCMAKE_BUILD_TYPE:STRING=Testing -DENABLE_TESTS:BOOL=ON
    - make
    - /usr/src/Slicer-build/python-install/bin/PythonSlicer -m pip install --upgrade pip
    - /usr/src/Slicer-build/python-install/bin/PythonSlicer -m pip install 2to3 lmfit
    - 2to3 -w /usr/src/Slicer-build/python-install/lib/python3.6/site-packages/uncertainties
    - Xvfb :99 -screen 0 1024x768x24 &> xvfb.log &
    - cd $CI_BUILDS_DIR/dosimetry4d-build
    - ctest -V

Do you mean adding SlicerElastix_DIR even when it is not there in the first place?
How to copy the s4ext file, shall I copy from web or do I need to create a local copy? There is already a folder of s4ext inside Slicer

lassoan · December 11, 2019, 4:21pm

Do you want to build extensions manually or make Slicer’s script build a list of extensions?

Alex_Vergara · December 12, 2019, 3:59am

Neither, I want Slicer to install required extensions automatically.

lassoan · December 12, 2019, 5:40am

Do you mean you bundle extensions with a custom Slicer build?

Alex_Vergara · December 12, 2019, 5:42am

I have made an extension that depends on another (Elastix) and I want that Slicer installs the required extension automatically. My extension shall be able to install in vanilla slicer-nightly.

lassoan · December 12, 2019, 5:55am

You need to specify the list of extensions yours depend on in the top-level CMakeLists.txt file (EXTENSION_DEPENDS variable). After this, you also need to regenerate the s4ext file and update it in ExtensionsIndex repository.

Alex_Vergara · December 13, 2019, 11:14am

I currently have (EXTENSION_DEPENDS variable)

May you please have a look to my extension and tell me what I am missing?

I am still having these errors

Alex_Vergara · December 19, 2019, 2:27pm

I still not able to force my extension to launch Elastix installation in tests

Alex_Vergara · March 2, 2020, 2:38pm

I have found a way to force install SlicerElastix from within python, however this requires Slicer Restart, this is not optimal, I wanted to do it from within the gitlab-ci itself but I get several errors:

Traceback (most recent call last):
  File "installElastix.py", line 2, in <module>
    emm = slicer.app.extensionsManagerModel()
AttributeError: module 'slicer' has no attribute 'app'
vtkDebugLeaks has found no leaks.

My installElastix.py script contains

import slicer
emm = slicer.app.extensionsManagerModel()
md = emm.retrieveExtensionMetadataByName('SlicerElastix')
emm.downloadAndInstallExtension(md['extension_id'])

In the environment there are

[Application]
path=<APPLAUNCHER_DIR>/bin/./SlicerApp-real
arguments=
name=Slicer
revision=e68a1a2
organizationDomain=www.na-mic.org
organizationName=NA-MIC

[ExtraApplicationToLaunch]

designer/shortArgument=
designer/help=Start Qt designer using Slicer plugins
designer/path=/opt/qt/5.11.2/gcc_64/bin/designer
designer/arguments=


[Environment]
additionalPathVariables=QT_PLUGIN_PATH,PYTHONPATH

[LibraryPaths]
1\path=<APPLAUNCHER_SETTINGS_DIR>/bin/.
2\path=../lib/Slicer-4.11/qt-loadable-modules
3\path=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/cli-modules/.
4\path=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/qt-loadable-modules/.
5\path=/usr/src/Slicer-build/OpenSSL
6\path=/usr/src/Slicer-build/python-install/lib
7\path=/usr/src/Slicer-build/VTK-build/lib/.
8\path=/usr/src/Slicer-build/teem-build/bin/.
9\path=/usr/src/Slicer-build/DCMTK-build/lib/.
10\path=/usr/src/Slicer-build/ITK-build/lib/.
11\path=/usr/src/Slicer-build/CTK-build/CTK-build/bin/.
12\path=/usr/src/Slicer-build/CTK-build/PythonQt-build/.
13\path=/usr/src/Slicer-build/LibArchive-install/lib
14\path=/usr/src/Slicer-build/JsonCpp-build/src/lib_json/.
15\path=/usr/src/Slicer-build/ParameterSerializer-build/lib/.
16\path=/usr/src/Slicer-build/SlicerExecutionModel-build/ModuleDescriptionParser/bin/.
17\path=/usr/src/Slicer-build/python-install/lib/python3.6/site-packages/numpy/core
18\path=/usr/src/Slicer-build/python-install/lib/python3.6/site-packages/numpy/lib
size=18

[Paths]
1\path=<APPLAUNCHER_SETTINGS_DIR>/bin/.
2\path=/opt/qt/5.11.2/gcc_64/bin
3\path=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/cli-modules/.
4\path=/usr/src/Slicer-build/python-install/bin
5\path=/usr/src/Slicer-build/teem-build/bin/.
size=5

[EnvironmentVariables]
SLICER_HOME=/usr/src/Slicer-build/Slicer-build
ITK_AUTOLOAD_PATH=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/ITKFactories/.
PIP_REQUIRE_VIRTUALENV=0
SSL_CERT_FILE=<APPLAUNCHER_SETTINGS_DIR>/share/Slicer-4.11/Slicer.crt
PYTHONHOME=/usr/src/Slicer-build/python-install
PYTHONNOUSERSITE=1



[QT_PLUGIN_PATH]
1\path=<APPLAUNCHER_SETTINGS_DIR>/bin
2\path=/usr/src/Slicer-build/CTK-build/CTK-build/bin
3\path=/opt/qt/5.11.2/gcc_64/plugins
size=3

[PYTHONPATH]
1\path=<APPLAUNCHER_SETTINGS_DIR>/bin/.
2\path=<APPLAUNCHER_SETTINGS_DIR>/bin/Python
3\path=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/qt-loadable-modules/.
4\path=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/qt-loadable-modules/Python
5\path=<APPLAUNCHER_SETTINGS_DIR>/lib/Slicer-4.11/qt-scripted-modules
6\path=/usr/src/Slicer-build/python-install/lib/python3.6
7\path=/usr/src/Slicer-build/python-install/lib/python3.6/lib-dynload
8\path=/usr/src/Slicer-build/python-install/lib/python3.6/site-packages
9\path=/usr/src/Slicer-build/VTK-build/lib/python3.6/site-packages
10\path=/usr/src/Slicer-build/CTK-build/CTK-build/bin/Python
11\path=/usr/src/Slicer-build/CTK-build/CTK-build/bin/.
size=11

Alex_Vergara · March 2, 2020, 4:15pm

I have tried to follow this circleci.yml in the slicer repository using the slicer-base docker image and it fails with

$ ../Slicer-build/BuildSlicer.sh
+ set -o pipefail
+ set -o
allexport       off
braceexpand     on
emacs           off
errexit         on
errtrace        off
functrace       off
hashall         on
histexpand      off
history         off
ignoreeof       off
interactive-comments    on
keyword         off
monitor         off
noclobber       off
noexec          off
noglob          off
nolog           off
notify          off
nounset         off
onecmd          off
physical        off
pipefail        on
posix           off
privileged      off
verbose         off
vi              off
xtrace          on
+ cd /usr/src/Slicer-build
+ /usr/bin/cmake -E make_directory /usr/src/Slicer
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-build
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix/tmp
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix/src
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-mkdir
+ cd /usr/src/Slicer-build/Slicer-prefix/src
+ /usr/bin/cmake -E echo_append
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-download
+ cd /usr/src/Slicer
+ /usr/bin/cmake -E echo_append
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-update
+ cd /usr/src/Slicer-build
+ /usr/bin/cmake -E echo_append
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-patch
+ cd /usr/src/Slicer-build/Slicer-build
+ /usr/bin/cmake -C/usr/src/Slicer-build/Slicer-prefix/tmp/Slicer-cache-Release.cmake -GNinja /usr/src/Slicer
loading initial cache file /usr/src/Slicer-build/Slicer-prefix/tmp/Slicer-cache-Release.cmake
CMake Error: The source directory "/usr/src/Slicer" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
+ exit 1
Authenticating with credentials from /Users/alexvergaragil/.docker/config.json
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x19196db]

lassoan · March 3, 2020, 2:48am

I would recommend to build SlicerExtensions project, which downloads, builds, and tests a list of extensions (by default, it takes the list from ExtensionsIndex repository, but you can provide your own list instead). This project takes care of dependencies between extensinos: it builds them in the right order, and also passes all needed build directories to extensions that depend on others. This is used every night to build all the extensions.

Alex_Vergara · March 4, 2020, 12:38pm

I am trying to replicate the behaviour of the circleCI docker instance but I was unsuccessful. see:

alex@CRCTcalcul:~$ docker run -it slicer/slicer-base  /bin/bash
Unable to find image 'slicer/slicer-base:latest' locally
latest: Pulling from slicer/slicer-base
a02a4930cb5d: Already exists
6a788e5d573f: Already exists
a9afa5c4a530: Already exists
6383f75ec3ac: Already exists
ec1353604d1f: Already exists
a6a7f5d80d82: Already exists
87bee5282cdd: Already exists
63309cde1def: Pull complete
Digest: sha256:4a69f356475c2974de6d35722474f674912300ee01ee50e3b8ff418bbc31865d
Status: Downloaded newer image for slicer/slicer-base:latest
[root@72bd75fc099d Slicer-build]# cd /usr/src/Slicer
bash: cd: /usr/src/Slicer: No such file or directory
[root@72bd75fc099d Slicer-build]# ../Slicer-build/BuildSlicer.sh
+ set -o pipefail
+ set -o
allexport      	off
braceexpand    	on
emacs          	off
errexit        	on
errtrace       	off
functrace      	off
hashall        	on
histexpand     	off
history        	off
ignoreeof      	off
interactive-comments	on
keyword        	off
monitor        	off
noclobber      	off
noexec         	off
noglob         	off
nolog          	off
notify         	off
nounset        	off
onecmd         	off
physical       	off
pipefail       	on
posix          	off
privileged     	off
verbose        	off
vi             	off
xtrace         	on
+ cd /usr/src/Slicer-build
+ /usr/bin/cmake -E make_directory /usr/src/Slicer
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-build
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix/tmp
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp
+ /usr/bin/cmake -E make_directory /usr/src/Slicer-build/Slicer-prefix/src
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-mkdir
+ cd /usr/src/Slicer-build/Slicer-prefix/src
+ /usr/bin/cmake -E echo_append
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-download
+ cd /usr/src/Slicer
+ /usr/bin/cmake -E echo_append
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-update
+ cd /usr/src/Slicer-build
+ /usr/bin/cmake -E echo_append
+ /usr/bin/cmake -E touch /usr/src/Slicer-build/Slicer-prefix/src/Slicer-stamp/Slicer-patch
+ cd /usr/src/Slicer-build/Slicer-build
+ /usr/bin/cmake -C/usr/src/Slicer-build/Slicer-prefix/tmp/Slicer-cache-Release.cmake -GNinja /usr/src/Slicer
loading initial cache file /usr/src/Slicer-build/Slicer-prefix/tmp/Slicer-cache-Release.cmake
CMake Error: The source directory "/usr/src/Slicer" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
+ exit 1
[root@72bd75fc099d Slicer-build]#

I want to create an image with Slicer extensions built by default.

Alex_Vergara · March 4, 2020, 1:58pm

Also, Is there a way to specify explicitly the list of extensions to make?

Topic		Replies	Views
Failing Tests - ImportError: No module named Elastix Development	11	825	June 18, 2019
Need to update extensions that other extensions may depend on Development	6	535	November 18, 2017
Extension contribution checklist Development	3	501	December 7, 2018
Building custom app with SegmentRegistration extension Development	2	191	July 28, 2023
Slicer elatix cannot be installed Support	8	698	February 21, 2019

Required extension not being installed in tests

Related topics