# Adding a CI build to a Slicer extension

**URL:** <https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857>\
**Category:** SlicerDMRI\
**Tags:** build\
**Created:** [July 28, 2023, 2:52pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857 "2023-07-28T14:52:17Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 28, 2023, 2:52pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/1 "2023-07-28T14:52:17Z")

</div>

Hi,  
I am trying to set up a GHA workflow for CI of the SlicerdMRI module:  
[ENH: Add GitHub actions build, test workflow file for CI by jhlegarreta · Pull Request #172 · SlicerDMRI/SlicerDMRI · GitHub](https://github.com/SlicerDMRI/SlicerDMRI/pull/172)

The CI is timing out due to Slicer taking beyond 6 hours to build, and not finishing:  
[ENH: Add GitHub actions build, test workflow file for CI · jhlegarreta/SlicerDMRI@8844bab · GitHub](https://github.com/jhlegarreta/SlicerDMRI/actions/runs/5685962029/job/15411909622#step:6:50076)

Maybe I’m missing many things, but I realize that:

- It compiles a number of extensions, or at least Slicer-dMRI, whose `HEAD` I am trying to build against Slicer, and thus it does not make sense to compile any default Slicer-dMRI extension. If all extensions being considered are using their `SuperBuild`, they will all be (re-)compiling all of Slicer and its dependencies. Is there a way to deactivate all extensions?  
e.g.  
[ENH: Add GitHub actions build, test workflow file for CI · jhlegarreta/SlicerDMRI@8844bab · GitHub](https://github.com/jhlegarreta/SlicerDMRI/actions/runs/5685962029/job/15411909622#step:6:294)
- It compiles both ITK and SimpleITK. Are both necessary for a minimal Slicer build?
- It builds the Python extensions. Is there a way to skip building Python extensions?
- Would using ninja instead of GNU be faster?

So, in general, what is the setting of flags for a minimal Slicer build within a reasonable time?

Thanks.

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [July 28, 2023, 3:09pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/2 "2023-07-28T15:09:38Z")

</div>

Yes, this issue is why we haven’t enable github actions for nightly builds or other testing. One idea is to build some of the prerequisites in their own github action repos and then store the build trees to share between stages, but it’s not really been investigated.

If you think you can make a simple Slicer build, you can turn off SimpleITK since it’s probably not used in SlicerDMRI. ITK is definitely needed, and it’s fairly fast. That may be enough, but probably not. Be sure to turn off BUILD\_TESTING too. You /might/ be able to turn off DICOM and SlicerDMRI might still work, but I’m not sure. You can probably turn off CLI modules for Slicer but that might carry over in to SlicerDMRI - not sure you can re-enable that in the extension build if it’s off in Slicer.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 28, 2023, 3:16pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/3 "2023-07-28T15:16:39Z")

</div>

Thanks for the answer Steve.

> you can turn off SimpleITK

> You /might/ be able to turn off DICOM

> You can probably turn off CLI modules

Where are the CMake flags documented so that I can do that?

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [July 28, 2023, 3:25pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/4 "2023-07-28T15:25:42Z")

</div>

You’ll just need to look in the main CMakeLists.txt.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 30, 2023, 2:19pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/5 "2023-07-30T14:19:22Z")

</div>

Note that for building an extension you don’t need to build Slicer, but you can start from a docker image that already has the complete Slicer build tree.

We already use CI for building Slicer for all pull requests, which starts from an image that has all the dependencies pre-built. Slicer build from that image fits in the time limit.

The plan was to set up an image that contains the fully built Slicer build tree (updated nightly), which could cut down time requirements even further. @jcfr had this been set up already?

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 30, 2023, 5:15pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/6 "2023-07-30T17:15:12Z")

</div>

> [@lassoan](#):
>
> Note that for building an extension you don’t need to build Slicer, but you can start from a docker image that already has the complete Slicer build tree.

Yes, that would be very helpful.

> [@lassoan](#):
>
> We already use CI for building Slicer for all pull requests, which starts from an image that has all the dependencies pre-built. Slicer build from that image fits in the time limit.

Yes, prior to submitting my PR to build SlicerDMRI I had a look at the Slicer CI folder, but found the workflow not straightforward to understand.

> [@lassoan](#):
>
> The plan was to set up an image that contains the fully built Slicer build tree (updated nightly), which could cut down time requirements even further.

Let me know when this is ready. Otherwise, I would still be interested in pulling an image that has all dependencies built. If an extension’s CI can do this easily, I could potentially use it.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 30, 2023, 7:11pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/7 "2023-07-30T19:11:25Z")

</div>

> [@jhlegarreta](#):
>
> Yes, prior to submitting my PR to build SlicerDMRI I had a look at the Slicer CI folder, but found the workflow not straightforward to understand.

The base image that comes with all Slicer depencies is this (updated nightly): [https://hub.docker.com/r/slicer/slicer-base/](https://hub.docker.com/r/slicer/slicer-base/)

> [@jhlegarreta](#):
>
> Let me know when this is ready. Otherwise, I would still be interested in pulling an image that has all dependencies built. If an extension’s CI can do this easily, I could potentially use it.

We would enable this for our extensions, too. Hopefully we’ll hear from @jcfr about this.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 30, 2023, 7:34pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/8 "2023-07-30T19:34:55Z")

</div>

> [@lassoan](#):
>
> The base image that comes with all Slicer depencies is this (updated nightly): [Docker](https://hub.docker.com/r/slicer/slicer-base/)

Yes, that I had seen. Thanks.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 30, 2023, 11:20pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/9 "2023-07-30T23:20:30Z")

</div>

The build at issue is warning about an extension/CLI module that is missing:

```auto
 Dependent extension UKFTractography cannot be found by CMake

```

Not sure if it is clear by the rest of the warning how the extension can be included/built: what is the appropriate CMake option to tell the Slicer build to download and compile a given extension/CLI module?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 30, 2023, 11:36pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/10 "2023-07-30T23:36:34Z")

</div>

If an extension depends on another extensions then you need to pass the build folder of each dependency in `<DependentExtensionName>_DIR` CMake variable.

Alternatively, you can put all s4ext files of the extensions you want to build into a folder and build this list of extensions as described [here](https://slicer.readthedocs.io/en/latest/developer_guide/extensions.html#build-list-of-extensions-manually).

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 30, 2023, 11:41pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/11 "2023-07-30T23:41:05Z")

</div>

FYI, I’ve built Slicer from `slicer/slicer-base` and pushed it to [`lassoan/slicer-built:20230730`](https://hub.docker.com/layers/lassoan/slicer-built/20230730/images/sha256-d33f9a9b8aedabfe83c1ffb2c5f7a698eb309a2a9269080e2fdd36666f5576e2?context=explore). It contains the full Slicer build tree, so it can be used for building any extension right away. It is quite big, though, so it may take a while for CI to download it. If it works then we could set up a repeating task to create a new image from latest Slicer source every night.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 30, 2023, 11:45pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/12 "2023-07-30T23:45:12Z")

</div>

> [@lassoan](#):
>
> If an extension depends on another extensions then you need to pass the build folder of each dependency in `<DependentExtensionName>_DIR` CMake variable.

OK, but this applies to the `SlicerDMRI` extension: how do I tell the Slicer build to download and build the `UKFTractography` extension? Do I need to proceed the exact same way as with `SlicerDMRI` by first cloning `UKFTractography`, then configuring, building? Luckily, it looks like the latter does not depend on other extensions.

> Alternatively, you can put all s4ext files of the extensions you want to build into a folder and build this list of extensions as described [here](https://slicer.readthedocs.io/en/latest/developer_guide/extensions.html#build-list-of-extensions-manually).

Not sure if I follow this, or how the linked page responds to the question above.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 30, 2023, 11:47pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/13 "2023-07-30T23:47:00Z")

</div>

Thanks for the effort. Will consider it at some point. For now, building Slicer from scratch is uncovering things that may need some attention, documenting, etc., so I would prefer to try to build it that way for now.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 30, 2023, 11:48pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/14 "2023-07-30T23:48:22Z")

</div>

> [@jhlegarreta](#):
>
> Do I proceed the exact same way as with `SlicerDMRI` by first cloning `UKFTractography`, the configuring, building?

Exactly. The inconvenience is that you need to run check out source code and build multiple extensions in the right order. If you simply [download the list of s4ext files](https://github.com/Slicer/ExtensionsIndex/) that you want to build and put it in a folder and run the single CMake command described [here](https://slicer.readthedocs.io/en/latest/developer_guide/extensions.html#build-list-of-extensions-manually) then it downloads and builds all the extensions automatically (in the correct order, taking into account dependencies between them).

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 31, 2023, 12:32am UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/15 "2023-07-31T00:32:38Z")

</div>

> [@jhlegarreta](#):
>
> For now, building Slicer from scratch is uncovering things that may need some attention, documenting, etc

To get started, it may be simpler to follow the provided instructions and defaults. If you change defaults then you will get to non-well-documented territory and may discover problems, which may not be the best use of your time. We attempt to fix all issues, but it is not necessary (and not possible) to test and fix _all_ the billions of combinations of build options.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 31, 2023, 1:26pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/16 "2023-07-31T13:26:15Z")

</div>

Andras, I am turning off modules or options that are available as options in CMake: if they are necessary to build Slicer, I am fine, but then they should not be made available as options in CMake.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 31, 2023, 1:44pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/17 "2023-07-31T13:44:08Z")

</div>

There are dozens of flags and of course not all combinations are tested and not all makes sense. Since it would be impractical to document (and maintain documentation) of all possible interplays between all options, we do not even attempt or plan to do this.

For example, turning off a Slicer core module is a very invasive action with unforeseeable consequences. There may be compile-time or runtime dependencies, there may be hard dependencies that would prevent some other modules from working at all, or absence of a module may just disable certain features in other modules. Some dependencies may be changed or removed by investing some work, if there is a good reason to do so. Even if we could explore and document some of the side effects of disabling a module in Slicer core, it is practically impossible to evaluate all the effects in all extensions continuously; or ask extension developers to perform this kind of impact analysis, document the results, and keep it up-to-date continuously.

Therefore, I would recommend to keep all options at the default value unless you have a strong reason to change one. Reducing build time or package size by changing default options may worth the effort for a well-funded commercial project, but probably not worth it for automated testing of an open-source extension.

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [July 31, 2023, 1:59pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/18 "2023-07-31T13:59:21Z")

</div>

It’s also worth noting, not just for Slicer but for any software, that configuration options that are not regularly tested should be considered experimental and/or expected not to work until proven. I’m sure that’s true for ITK and VTK for example, and I’m sure we’ve all encountered that. For Slicer this means that the configuration used on the factory machines with the documented build systems is the starting point for any experiments.

Perhaps we should make this statement explicit in the build documentation. We could even generate a cmake warning whenever a configuration is changed such that the build is going down and untested pathway.

If there are configurations that are important for key use cases then we should set up CI to keep them tested and working.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 31, 2023, 2:03pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/19 "2023-07-31T14:03:05Z")

</div>

Agreed Steve: to me, the point is that if a module is part of the core, not sure why they should be optional. An intermediate step would be maybe to mark them as `Advanced` variables or grouped under a `Core` category.

Above all, I am grateful for all the work that you and the community has put and puts into this, and I am more than happy to continue helping, needless to say this. I have good reasons to set up a CI on GitHub for the extension at issue. We can discuss this on a call if necessary.

---

<div class="post-metadata">

**Author:** ![jhlegarreta](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jhlegarreta/32/66542_2.png) [@jhlegarreta](https://discourse.slicer.org/u/jhlegarreta)\
**Post date:** [July 31, 2023, 2:07pm UTC](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857/20 "2023-07-31T14:07:30Z")

</div>

Thanks for the pointers. Had a look at this.

According to the extension index, `SlicerDMRI` seems not to depend on any other extension/module according to its extension index file:  
[https://github.com/Slicer/ExtensionsIndex/blob/main/SlicerDMRI.s4ext#L5](https://github.com/Slicer/ExtensionsIndex/blob/main/SlicerDMRI.s4ext#L5)

I now see that the configuration process, a few lines later,  
[ENH: Add GitHub actions build, test workflow file for CI · jhlegarreta/SlicerDMRI@67e378d · GitHub](https://github.com/jhlegarreta/SlicerDMRI/actions/runs/5707614346/job/15464465415#step:7:186)

says:

```auto
-- Setting EXTENSION_DEPENDS ...................: UKFTractography

```

Which I assume stems from the extension’s `CMakeLists.txt`: [https://github.com/SlicerDMRI/SlicerDMRI/blob/610533480896d2884cb9ab0671c88874d237247d/CMakeLists.txt#L14](https://github.com/SlicerDMRI/SlicerDMRI/blob/610533480896d2884cb9ab0671c88874d237247d/CMakeLists.txt#L14)

So there seems to be mismatch between what the extension states as its dependencies in its `CMakeLists.txt` file and the extension index `s4ext` file recipe.

Is there an automated way to keep these in sync?

[Next page](https://discourse.slicer.org/t/adding-a-ci-build-to-a-slicer-extension/30857.md?page=2)