Maybe I’m missing many things, but I realize that:
It compiles a number of extensions, or at least Slicer-dMRI, whose HEAD I am trying to build against Slicer, and thus it does not make sense to compile any default Slicer-dMRI extension. If all extensions being considered are using their SuperBuild, they will all be (re-)compiling all of Slicer and its dependencies. Is there a way to deactivate all extensions?
e.g. ENH: Add GitHub actions build, test workflow file for CI · jhlegarreta/SlicerDMRI@8844bab · GitHub
It compiles both ITK and SimpleITK. Are both necessary for a minimal Slicer build?
It builds the Python extensions. Is there a way to skip building Python extensions?
Would using ninja instead of GNU be faster?
So, in general, what is the setting of flags for a minimal Slicer build within a reasonable time?
Yes, this issue is why we haven’t enable github actions for nightly builds or other testing. One idea is to build some of the prerequisites in their own github action repos and then store the build trees to share between stages, but it’s not really been investigated.
If you think you can make a simple Slicer build, you can turn off SimpleITK since it’s probably not used in SlicerDMRI. ITK is definitely needed, and it’s fairly fast. That may be enough, but probably not. Be sure to turn off BUILD_TESTING too. You /might/ be able to turn off DICOM and SlicerDMRI might still work, but I’m not sure. You can probably turn off CLI modules for Slicer but that might carry over in to SlicerDMRI - not sure you can re-enable that in the extension build if it’s off in Slicer.
Note that for building an extension you don’t need to build Slicer, but you can start from a docker image that already has the complete Slicer build tree.
We already use CI for building Slicer for all pull requests, which starts from an image that has all the dependencies pre-built. Slicer build from that image fits in the time limit.
The plan was to set up an image that contains the fully built Slicer build tree (updated nightly), which could cut down time requirements even further. @jcfr had this been set up already?
Yes, prior to submitting my PR to build SlicerDMRI I had a look at the Slicer CI folder, but found the workflow not straightforward to understand.
Let me know when this is ready. Otherwise, I would still be interested in pulling an image that has all dependencies built. If an extension’s CI can do this easily, I could potentially use it.
The build at issue is warning about an extension/CLI module that is missing:
Dependent extension UKFTractography cannot be found by CMake
Not sure if it is clear by the rest of the warning how the extension can be included/built: what is the appropriate CMake option to tell the Slicer build to download and compile a given extension/CLI module?
If an extension depends on another extensions then you need to pass the build folder of each dependency in <DependentExtensionName>_DIR CMake variable.
Alternatively, you can put all s4ext files of the extensions you want to build into a folder and build this list of extensions as described here.
FYI, I’ve built Slicer from slicer/slicer-base and pushed it to lassoan/slicer-built:20230730. It contains the full Slicer build tree, so it can be used for building any extension right away. It is quite big, though, so it may take a while for CI to download it. If it works then we could set up a repeating task to create a new image from latest Slicer source every night.
OK, but this applies to the SlicerDMRI extension: how do I tell the Slicer build to download and build the UKFTractography extension? Do I need to proceed the exact same way as with SlicerDMRI by first cloning UKFTractography, then configuring, building? Luckily, it looks like the latter does not depend on other extensions.
Alternatively, you can put all s4ext files of the extensions you want to build into a folder and build this list of extensions as described here.
Not sure if I follow this, or how the linked page responds to the question above.
Thanks for the effort. Will consider it at some point. For now, building Slicer from scratch is uncovering things that may need some attention, documenting, etc., so I would prefer to try to build it that way for now.
Exactly. The inconvenience is that you need to run check out source code and build multiple extensions in the right order. If you simply download the list of s4ext files that you want to build and put it in a folder and run the single CMake command described here then it downloads and builds all the extensions automatically (in the correct order, taking into account dependencies between them).
To get started, it may be simpler to follow the provided instructions and defaults. If you change defaults then you will get to non-well-documented territory and may discover problems, which may not be the best use of your time. We attempt to fix all issues, but it is not necessary (and not possible) to test and fix all the billions of combinations of build options.
Andras, I am turning off modules or options that are available as options in CMake: if they are necessary to build Slicer, I am fine, but then they should not be made available as options in CMake.
There are dozens of flags and of course not all combinations are tested and not all makes sense. Since it would be impractical to document (and maintain documentation) of all possible interplays between all options, we do not even attempt or plan to do this.
For example, turning off a Slicer core module is a very invasive action with unforeseeable consequences. There may be compile-time or runtime dependencies, there may be hard dependencies that would prevent some other modules from working at all, or absence of a module may just disable certain features in other modules. Some dependencies may be changed or removed by investing some work, if there is a good reason to do so. Even if we could explore and document some of the side effects of disabling a module in Slicer core, it is practically impossible to evaluate all the effects in all extensions continuously; or ask extension developers to perform this kind of impact analysis, document the results, and keep it up-to-date continuously.
Therefore, I would recommend to keep all options at the default value unless you have a strong reason to change one. Reducing build time or package size by changing default options may worth the effort for a well-funded commercial project, but probably not worth it for automated testing of an open-source extension.
It’s also worth noting, not just for Slicer but for any software, that configuration options that are not regularly tested should be considered experimental and/or expected not to work until proven. I’m sure that’s true for ITK and VTK for example, and I’m sure we’ve all encountered that. For Slicer this means that the configuration used on the factory machines with the documented build systems is the starting point for any experiments.
Perhaps we should make this statement explicit in the build documentation. We could even generate a cmake warning whenever a configuration is changed such that the build is going down and untested pathway.
If there are configurations that are important for key use cases then we should set up CI to keep them tested and working.
Agreed Steve: to me, the point is that if a module is part of the core, not sure why they should be optional. An intermediate step would be maybe to mark them as Advanced variables or grouped under a Core category.
Above all, I am grateful for all the work that you and the community has put and puts into this, and I am more than happy to continue helping, needless to say this. I have good reasons to set up a CI on GitHub for the extension at issue. We can discuss this on a call if necessary.
So there seems to be mismatch between what the extension states as its dependencies in its CMakeLists.txt file and the extension index s4ext file recipe.