Packaging SlicerVMTK extension with custom Slicer build

mschumaker · August 2, 2018, 7:49pm

I have added a Slicer_EXTENSION_SOURCE_DIRS entry to my CMakeCache.txt for a Slicer build (Aug 1 nightly). I have entered the source directory for one extension, the SlicerExtension-VMTK, which depends on the external VMTK package. The build error I’m getting is that it can’t find the External_VMTK.cmake file.

What appears to be happening is the extension’s SuperBuild.cmake file calls a macro, ExternalProject_Include_Dependencies. I’ve searched, and there is a file that contains this macro in Slicer/CMake/ExternalProjectDependency.cmake, but that’s not the macro that gets called. Instead, it looks like it picks up the macro in the BRAINSTools extension, in Slicer-SuperBuild/BRAINSTools/CMake/ExternalProjectDependency.cmake. These files are very different.

The macro starts around line 450 of the file in BRAINSTools, and the error message I get is on line 639. There are three places it looks for the External_VMTK.cmake file:

/workingdir/Slicer/SuperBuild/External_VMTK.cmake
(blank)
/External_VMTK.cmake

The correct location is /workingdir/Extensions/SlicerExtension-VMTK/SuperBuild.

My questions are: which version of the macro should it be using? Is it calling the wrong one? Would the version of the macro in Slicer/CMake be able to pick up the correct location for the External_VMTK.cmake file? Is there a way to correct this?

Thanks very much. The full text of the error I get is below.

– Configuring extension directory: SlicerExtension-VMTK
CMake Warning (dev) at /short/Extensions/SlicerExtension-VMTK/CMakeLists.txt:4 (project):
Policy CMP0048 is not set: project() command manages VERSION variables.
Run “cmake --help-policy CMP0048” for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

The following variable(s) would be set to empty:

PROJECT_VERSION
PROJECT_VERSION_MAJOR
PROJECT_VERSION_MINOR
PROJECT_VERSION_PATCH

This warning is for project developers. Use -Wno-dev to suppress it.

– Checking EXTENSION_NAME variable
– Checking EXTENSION_NAME variable - NOTDEFINED
– Checking MODULE_NAME variable
– Checking MODULE_NAME variable - NOTDEFINED
– Checking PROJECT_NAME variable
– Checking PROJECT_NAME variable - SlicerVMTK
– Setting EXTENSION_NAME …: SlicerVMTK
– SuperBuild - First pass
CMake Error at /short/Slicer-SuperBuild/BRAINSTools/CMake/ExternalProjectDependency.cmake:639 (message):
Can’t find External_VMTK.cmake

Call Stack (most recent call first):
/short/Extensions/SlicerExtension-VMTK/SuperBuild.cmake:32 (ExternalProject_Include_Dependencies)
/short/Extensions/SlicerExtension-VMTK/CMakeLists.txt:33 (include)

– Configuring incomplete, errors occurred!
See also “/short/Slicer-SuperBuild/Slicer-build/CMakeFiles/CMakeOutput.log”.
See also “/short/Slicer-SuperBuild/Slicer-build/CMakeFiles/CMakeError.log”.
make[2]: *** [Slicer-prefix/src/Slicer-stamp/Slicer-configure] Error 1
make[1]: *** [CMakeFiles/Slicer.dir/all] Error 2
make: *** [all] Error 2

mschumaker · August 2, 2018, 8:01pm

The SlicerExtension-VMTK code I’m using is at https://github.com/mschumak/SlicerExtension-VMTK.git commit 550c2813a756d8bec0bb5b8286ed68afc93860f2.

mschumaker · August 7, 2018, 9:55pm

Perhaps I’ll start with a more basic question. What is the best way to package an extension with a custom build? I’ve tried doing it by setting a Slicer_EXTENSION_SOURCE_DIRS CMake value before building, and have encountered problems, though these may be specific to the SlicerVMTK extension. Should I continue approaching it this way?

jcfr · August 7, 2018, 10:07pm

Yes.

Please, report here the errors you encountered.

mschumaker · August 8, 2018, 8:54pm

Thank you. I’ve set up a project using cookiecutter (thanks again for help using it). Before building, I added Slicer_EXTENSION_SOURCE_DIRS to the CMakeCache.
Right now, I’m trying to get my own scripted modules to link with the application. I specified a directory with my extension, which was set up using the ExtensionWizard. It has 10 modules. An error occurs when it reads the CMakeLists file for a module. It may not recognize the slicerMacroBuildScriptedModule macro?

– Configuring extension directory: PADFusion
CMake Warning (dev) at /Users/michaelschumaker/Development/mschumaker/Slicelets/PADFusion/PADFusion/CMakeLists.txt:3 (project):
Policy CMP0048 is not set: project() command manages VERSION variables.
Run “cmake --help-policy CMP0048” for policy details. Use the cmake_policy
command to set the policy and suppress this warning.

The following variable(s) would be set to empty:
PROJECT_VERSION
PROJECT_VERSION_MAJOR
PROJECT_VERSION_MINOR
PROJECT_VERSION_PATCH
This warning is for project developers. Use -Wno-dev to suppress it.

– Checking EXTENSION_NAME variable
– Checking EXTENSION_NAME variable - NOTDEFINED
– Checking MODULE_NAME variable
– Checking MODULE_NAME variable - NOTDEFINED
– Checking PROJECT_NAME variable
– Checking PROJECT_NAME variable - PADFusion
– Setting EXTENSION_NAME …: PADFusion
CMake Error at /Users/michaelschumaker/Development/mschumaker/Slicelets/PADFusion/PADFusion/FusionWorkflow/CMakeLists.txt:14 (slicerMacroBuildScriptedModule):
Unknown CMake command “slicerMacroBuildScriptedModule”.

– Configuring incomplete, errors occurred!
See also “/short/appTemplate/PADPlanner-SuperBuild/Slicer-build/CMakeFiles/CMakeOutput.log”.
See also “/short/appTemplate/PADPlanner-SuperBuild/Slicer-build/CMakeFiles/CMakeError.log”.
make[2]: *** [slicersources-build/Slicer-prefix/src/Slicer-stamp/Slicer-configure] Error 1
make[1]: *** [slicersources-build/CMakeFiles/Slicer.dir/all] Error 2
make: *** [all] Error 2

mschumaker · August 8, 2018, 8:59pm

Noticed I was still working within the cookiecutter virtual environment we discussed here https://github.com/KitwareMedical/SlicerCustomAppTemplate/issues/5. I’m trying it again from my regular shell.

Same error.

jcfr · August 8, 2018, 9:14pm

Unknown CMake command “slicerMacroBuildScriptedModule

This is most likely explained by the fact Slicer_USE_PYTHONQT is disabled by default in the template.

After integrating the following PR a more obvious error message will be reported:

mschumaker · August 8, 2018, 9:18pm

Thanks. Yes, that will make it easier to figure out why it failed. I’ll turn Slicer_USE_PYTHONQT on.

jcfr · August 8, 2018, 9:19pm

Before building, I added Slicer_EXTENSION_SOURCE_DIRS

Also, the download of the module or extension source can be automatic using an approach similar to this one:

github.com

KitwareMedical/SlicerCustomAppTemplate/blob/c463c1aed20a2ede2bb741533404bd0c09d1530e/{{cookiecutter.project_name}}/CMakeLists.txt#L115-L125


      
          # SlicerOpenIGTLink
          #set(extension_name "SlicerOpenIGTLink")
          #set(${extension_name}_SOURCE_DIR "${CMAKE_BINARY_DIR}/${extension_name}")
          #FetchContent_Populate(${extension_name}
          #  SOURCE_DIR     ${${extension_name}_SOURCE_DIR}
          #  GIT_REPOSITORY ${EP_GIT_PROTOCOL}://github.com/openigtlink/SlicerOpenIGTLink.git
          #  GIT_TAG        2b92f7b1ffe02403109b671f28424e8770e902a0
          #  GIT_PROGRESS   1
          #  QUIET
          #  )
          #list(APPEND Slicer_EXTENSION_SOURCE_DIRS ${${extension_name}_SOURCE_DIR})

mschumaker · August 10, 2018, 4:11pm

I’m making progress with an application started with cookiecutter, and including my scripted modules. One question I have is about additional files for a module. I have some tables in plist files in the Resources directory of a scripted module. How do I make sure these will be copied to the qt-scripted-modules/Resources directory when building?

Another minor thing I noticed is a problem with the version numbers and dates that appear in the compiled application. In the Mac menu bar, the version number and date are for Slicer, while at the top of the application window, the version number is what I defined, but the date is all zeros. I’ve included an image. Is this a bug, or is there something I need to do?
Thanks!

jcfr · August 10, 2018, 4:54pm

This is because you do not yet have a git repository associated with your custom application source code. In that case, the there is now way to know the date associated with the last commit.

This is indeed a bug, for now the following lines need to be changed to use Slicer_MAIN_PROJECT_VERSION_FULL. (Note that ideally it should SLICER_APP_VERSION_FULL but this is not yet possible because repository info are only extracted for Slicer itself and the main application only)

Could you create a Slicer PR to address this ? Thanks

https://github.com/Slicer/Slicer/blob/ad3fcd4ddf367363f9fb258b6dc2fcee06cd2039/CMake/SlicerMacroBuildApplication.cmake#L449-L450

mschumaker · August 10, 2018, 6:57pm

Ok, I’ve submitted a PR with those lines changed.

Ok, thanks. I put the application code in a sub-directory of a project, so the .git directory is two levels up. I’ll reorganize things once I have a better idea what I’m doing.

mschumaker · August 10, 2018, 7:50pm

I also had this question:

Thanks!

jcfr · August 10, 2018, 7:58pm

Thanks for the contribution. PR#1003 has just been integrated as 27343

What do you mean exactly by some tables in plist files ?

There are two approaches for associated resources with scripted module:

You can associate resources with scripted module by listing them as RESOURCES. For example, see DataProbe. See also you previous post: Adding an icon to a button - #15 by mschumaker
You can generate a Qt resource file. Also already discussed in Adding an icon to a button - #15 by mschumaker

mschumaker · August 10, 2018, 8:15pm

Oh… the answer was in my own previous post!
I hadn’t added the files to the MODULE_PYTHON_RESOURCES list.
Property list (.plist) files are an XML-based data format on Mac/iOS. Python has a plistlib library to read them. I’m using some color look-up tables that are stored that way.

jcfr · August 10, 2018, 8:22pm

Property list (.plist) files are an XML-based data format on Mac/iOS. Python has a plistlib library to read them.

I see. The Slicer build system currently provides its one template for the plist file integrated in the Slicer.app (or `Custom.app):

https://github.com/Slicer/Slicer/blob/87e8a89274eca8c0a249df8785ded16a060006ab/CMake/SlicerMacroBuildApplication.cmake#L451

This was done back in 2014 to ensure the application would work with High DPI monitor. See https://github.com/Slicer/Slicer/commit/a913ea9af8705fc766d5acd7d9689a8202ba2310

If really needed, we could provide custom application with a way to provide their own template, would that help you ?

That said, using a more traditional approach would ensure the work you do is portable on windows, linux, …

mschumaker · August 10, 2018, 9:27pm

Thanks, that’s probably more work than necessary. Reading them isn’t a problem, it’s that I forgot that I should include them in the CMakeLists file. I just tested that, and it worked.
I’m finding the files by starting from the path of the module’s Python script, then the function plistlib.readPlist reads a file and generates a dictionary. It should still be portable to other systems. Here’s what I’m doing:

    self.moduleDir = os.path.dirname(slicer.util.modulePath(self.__module__))
    ...
    clutDir = "Resources/CLUT"
    NIHFile = "NIH.plist"
    NIHFullPath = self.moduleDir+"/"+clutDir+"/"+NIHFile

    import plistlib
    plistOutput = plistlib.readPlist(NIHFullPath)
    plistRed = plistOutput['Red']
    plistGreen = plistOutput['Green']
    plistBlue = plistOutput['Blue']

Then I build a vtkMRMLColorTableNode from these arrays.

mschumaker · August 15, 2018, 2:58pm

Thanks for your previous assistance.
I’m back to trying to get SlicerVMTK to build with my application. I’m working from an application started using cookiecutter. The build is able to identify a dependency on VMTK and build it, but when it gets to building SlicerVMTK via the ExternalProjectDependency.cmake script, it can’t find SuperBuild/External_VMTK.cmake. The line that produces the error message I get is at:

github.com

Slicer/Slicer/blob/main/CMake/ExternalProjectDependency.cmake#L858


      
            #message(${_sb_proj} "[${_sb_proj}] Property SB_${dep}_OPTIONAL set to ${_optional}")
            if(NOT _optional)
              list(APPEND _sb_REQUIRED_DEPENDS ${dep})
            endif()
          endforeach()
          
          # Display dependency of project being processed
          if(_sb_REQUIRED_DEPENDS AND SB_SECOND_PASS AND ${_sb_SB_VAR})
            set(dependency_str "")
            foreach(dep ${_sb_REQUIRED_DEPENDS})
              get_property(_is_included GLOBAL PROPERTY SB_${dep}_FILE_INCLUDED)
              set(_include_status "")
              if(_is_included)
                set(_include_status "[INCLUDED]")
              endif()
              set(dependency_str "${dependency_str}${dep}${_include_status}, ")
            endforeach()
            ExternalProject_Message(${_sb_proj} "${_sb_proj} => Requires ${dependency_str}")
          endif()
          
          # Save variables

The error I get is:

– Setting EXTENSION_NAME …: SlicerVMTK
– SuperBuild - First pass
CMake Error at /short/pack/PADPlanner-SB/slicersources-src/CMake/ExternalProjectDependency.cmake:858 (message):
Can’t find External_VMTK.cmake
Call Stack (most recent call first):
/short/pack/PADPlanner-SB/SlicerExtension-VMTK/SuperBuild.cmake:32 (ExternalProject_Include_Dependencies)
/short/pack/PADPlanner-SB/SlicerExtension-VMTK/CMakeLists.txt:33 (include)

I’m fetching the extension code from a repository, and so the extension source is being placed in SlicerExtension-VMTK in the main project-SuperBuild directory. I need help with how to get the build script to find the External_VMTK.cmake file.

mschumaker · August 15, 2018, 5:58pm

It looks like when the build is at the stage of configuring the extension directory for SlicerExtension-VMTK, the value of EXTERNAL_PROJECT_DIR is /short/pack/PADPlanner-SB/slicersources-src/SuperBuild/External_VMTK.cmake, rather than the location of the extension.

mschumaker · August 17, 2018, 4:30pm

@lassoan, @jcfr, any thoughts on this? I may test another extension, because I’m not sure if this is a general problem with including extensions that have dependencies via Slicer_EXTENSION_SOURCE_DIRS, or just with SlicerVMTK. I looked at the elastix extension to see if I could see something SlicerVMTK was missing, and it looked about the same, but apparently that wasn’t the best choice for comparison.

Topic		Replies	Views
SlicerExtension-VMTK build failing on MacOS 10.12, VTKv9, Qt 5.10 Development extensions-manager , vmtk	26	1614	July 20, 2018
CMake error in SlicerExtensionCPack.cmake while configuring extension Support extensions , cmake	1	289	October 6, 2023
SlicerRT extension build failure Development	4	453	May 30, 2018
Failures due to build order of Slicer CAT with superbuild remote modules Support	12	1003	February 9, 2022
Extension dependencies in custom apps Development build	7	411	May 24, 2022

Packaging SlicerVMTK extension with custom Slicer build

Related topics