Nightly build TESTS failing for extensions with dependencies to other scriptable modules

Development
41

By “module” I mean Slicer scripted module. You can use any Python packages that are independent from Slicer, in any way you want. You just have to place them in a subdirectory of any Slicer module directory so that Python can find it.

42

I think in the case of @che85, the issue comes from the dependency on a Slicer extension, SlicerDevelopmentToolbox, that contains a set of python packages to reuse from multiple Slicer modules, and it is not loaded when it is needed. It sounds like we would need to add “fake” widget/logic to the classes from that extension to make it loaded in order. Christian, please correct me if I am wrong.

43

It doesn’t need to be an actual module, the only thing that needs to be ensured is that the python files are deployed to the proper location, which is recognized by the extension. An example can be DICOMLib or Modules/Loadable/Segmentations/EditorEffects/Python.

If the files are deployed to the right place, the dependencies need to be added (at least this is how I managed to get it to work). See the git commits I referenced above.

44

@cpinter It would be great if that works. In my opinion Slicer should set those in a generic manner instead of having those attributes set within the extensions CMakeLists. I think otherwise it would be very error prone and a lot of overhead.

No idea if that can be done in Slicer itself.

1 Like
45

Hi,

You can build the project Slicer/extensions/Cmake.

Look for “build extensionindex” on the devel FAQ.

Hth
Jc

46

@jcfr That’s what I did and those are the results: http://slicer.cdash.org/testDetails.php?test=8443330&build=11357232

47

@che85 you can confirm this with otool -l LIBRARYNAME.

There is a hack we could use, essentially to daisy-chain dependencies, but at that level of effort we’re better off reducing the number of shared libraries.

(last time I looked at the slow startup issue on macOS – especially debug build – I think there were over 600 shared libraries loaded, and majority of them were ours. IIRC there were something like 30 million strcmps during startup while sorting symbols).

48

FWIW, a patch is being integrated into VTK to get ride of the PythonD
library and remove the number of libraries.

We could backport it.

Jc

1 Like
49

@ihnorton Here is the output of otool

50

Update: Those path additions that enabled the tests to run locally didn’t do the trick on the factory.
http://slicer.cdash.org/testDetails.php?test=8445508&build=1136268
The only reason I can see is that the *_DIR paths are not correct. I’ll check on our factory machine.

51

Hi guys,

are there any updates/ideas for solving this issue?

52

Considering the switch to Qt5 and other dramatic changes in the core, it is unfortunate that we don’t have a reliable way to test extensions to identify regressions associated with that process.

53

To clarify, extensions are still build and tested nightly. It means you will be able to look at the build report every day.

54

@jcfr not sure I understand. My understanding was that the issue is “Nightly build TESTS failing for extensions with dependencies to other scriptable modules”, which means we have false alarms instead of actually checking for regressions in the code. Am I missing something?

1 Like
55

@che85 is there a bug report that documents the problem? If not, can you submit one?

56

I created a bug report for this issue. See https://issues.slicer.org/view.php?id=4498

1 Like
57

Seems like the tests are running now. I added the following lines to CMakeLists.txt

and

http://slicer.cdash.org/testDetails.php?test=8625898&build=1194275

The find_package commands are commented out because they caused CMake configure errors, which in my opinion is giving misleading information to the developer:

http://slicer.cdash.org/viewConfigure.php?buildid=1193166

For any reason when using those find_package commands, DCMQI is not found. @jcfr I feel like the following warnings are always displayed. Is that correct?

CMake Warning at /home/kitware/Dashboards/Nightly/Slicer-0/Extensions/CMake/SlicerBlockAdditionalLauncherSettings.cmake:48 (message):
  Dependent extension PETDICOMExtension cannot be found by CMake
  find_package(), therefore paths variables cannot be imported from this
  extension.  The problem can be resolved by generating
  PETDICOMExtensionConfig.cmake by adding
  include(${Slicer_EXTENSION_GENERATE_CONFIG}) to the top-level
  CMakeLists.txt of the dependent exension.
1 Like
58

Looking at the command line of [1] I am also wondering, why ${SlicerDevelopmentToolbox_DIR} could be resolved to /Users/kitware/Dashboards/Nightly/S-481-E-b/SlicerDevelopmentToolbox-build/, but neither${PETDICOMExtension_DIR} could be resolved --> PETDICOMExtension_DIR-NOTFOUND nor ${DCMQI_DIR} could be resolved --> DCMQI_DIR-NOTFOUND.

I will run some more tests.

[1] http://slicer.cdash.org/testDetails.php?test=8625898&build=1194275

59

@jcfr @lassoan Can you explain why those messages still appear even though I included the mentioned files for every single project and built each one on its own? Am I missing something? Configuring only works, if I specify the directories manually.

CMake Warning at /Users/christian/sources/cpp/Slicer/Extensions/CMake/SlicerBlockAdditionalLauncherSettings.cmake:48 (message):
  Dependent extension SlicerDevelopmentToolbox cannot be found by CMake
  find_package(), therefore paths variables cannot be imported from this
  extension.  The problem can be resolved by generating
  SlicerDevelopmentToolboxConfig.cmake by adding
  include(${Slicer_EXTENSION_GENERATE_CONFIG}) to the top-level
  CMakeLists.txt of the dependent exension.
Call Stack (most recent call first):
  /Users/christian/D/S4R/Slicer-build/UseSlicer.cmake:281 (include)
  CMakeLists.txt:19 (include)


CMake Warning at /Users/christian/sources/cpp/Slicer/Extensions/CMake/SlicerBlockAdditionalLauncherSettings.cmake:48 (message):
  Dependent extension DCMQI cannot be found by CMake find_package(),
  therefore paths variables cannot be imported from this extension.  The
  problem can be resolved by generating DCMQIConfig.cmake by adding
  include(${Slicer_EXTENSION_GENERATE_CONFIG}) to the top-level
  CMakeLists.txt of the dependent exension.
Call Stack (most recent call first):
  /Users/christian/D/S4R/Slicer-build/UseSlicer.cmake:281 (include)
  CMakeLists.txt:19 (include)


CMake Warning at /Users/christian/sources/cpp/Slicer/Extensions/CMake/SlicerBlockAdditionalLauncherSettings.cmake:48 (message):
  Dependent extension PETDICOMExtension cannot be found by CMake
  find_package(), therefore paths variables cannot be imported from this
  extension.  The problem can be resolved by generating
  PETDICOMExtensionConfig.cmake by adding
  include(${Slicer_EXTENSION_GENERATE_CONFIG}) to the top-level
  CMakeLists.txt of the dependent exension.
Call Stack (most recent call first):
  /Users/christian/D/S4R/Slicer-build/UseSlicer.cmake:281 (include)
  CMakeLists.txt:19 (include)
60

To summarize, this last issue was due to missing dependencies and missing config files generation in the extensions. The following commits address the problem: Slicer/ExtensionsIndex@34a0896 and Slicer/ExtensionsIndex@3bdb20a