# How can I subscribe to monitor individual extensions on Slicer dashboard?

**URL:** <https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261>\
**Category:** Development\
**Tags:** dashboard\
**Created:** [March 7, 2018, 3:57pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261 "2018-03-07T15:57:25Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 7, 2018, 3:57pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/1 "2018-03-07T15:57:25Z")

</div>

@jcfr [mentioned](https://github.com/QIICR/dcmqi/issues/335#issuecomment-369654159)

> The good news is that CDash support email notification and now support specifying more than one repository.

Can someone help me figure out how to do this? My guess was that this should be configured in project subscription, but I don’t see how this can be done. There are no labels to select from.

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/2/5/25a332369c55f06fc2a43cd67469c65531872008.png)

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 7, 2018, 5:10pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/2 "2018-03-07T17:10:38Z")

</div>

Hi @fedorov ,

When using `ctest` and `CDash` to build a single project, this work our of the box.

The challenge here is that we conveniently centralize the configure/build/test of more than hundred of projects.

You will find below comments and idea I came up with while looking at the current system

### in a nutshell

… the extension build system is doing the following:

1. Discover and parse all extension description files
2. Sort extension topologically based on dependency information
3. Add one external project per extension with associated dependency information (this allow to support parallel build): This is used to download the source code and build the project.

The external project is added [here](https://github.com/Slicer/Slicer/blob/34cd89b97998bc0d23eaebcebc3b28bb5d6c689a/Extensions/CMake/SlicerBlockBuildPackageAndUploadExtensions.cmake#L255-L263).

### way forward

The idea is to have for every extension build, repository _information_ send along the configure/build/test restults. This should allow to associate together new configure/build/test error or warning with a given revision of the code.

To support this, I think we need to:

1. keep around the extension source directory between successive builds
2. ensure that the command [ctest\_update](https://cmake.org/cmake/help/latest/command/ctest_update.html) is called.

### Do we need to keep extension source checkout around between build ?

Say differently, how does CTest/CDash find out the _diff_ associated with two consecutive configure/build/test ?

**Note 1:** the _diff_ is important to know as it allows to find out the list of authors who potentially introduced warnings/errors/failures/… and later notify them.

**Note 2:** the extension sources are checked out cleanly every night, we are currently not keep source trees around. The only source tree we currently keep around it the one associated with the [ExtensionsIndex](https://github.com/Slicer/ExtensionsIndex/)

The answer to this question will help us move forward and decide what are the steps.

#### case 1: if the answer is no

This means that CDash can compute the _diff_ on its his own by only using the build name and repository name.

In that case, it may be as simple as adding the `ctest_update` call between these two calls:

> <https://github.com/Slicer/Slicer/blob/34cd89b97998bc0d23eaebcebc3b28bb5d6c689a/Extensions/CMake/SlicerBlockBuildPackageAndUploadExtension.cmake#L133-L134>

**intuition:** I doubt this is the case, I will confirm this with the CDash development team.

### case 2: if the answer is yes

This means that the _diff_ is generated locally by using the call to `ctest_update`

In that case, I think more changes would be involved:

We need to:

1. update the system to keep `extension source checkouts` outside of the `extension index build tree`
2. use `ctest_update` to update the checkout instead of teaching ExternalProject how to do it. The code used to setup the download command is [here](https://github.com/Slicer/Slicer/blob/34cd89b97998bc0d23eaebcebc3b28bb5d6c689a/Extensions/CMake/SlicerBlockBuildPackageAndUploadExtensions.cmake#L132-L213)

Setting either [CTEST\_GIT\_UPDATE\_OPTIONS](https://cmake.org/cmake/help/latest/variable/CTEST_GIT_UPDATE_OPTIONS.html#variable:CTEST_GIT_UPDATE_OPTIONS) or [CTEST\_GIT\_UPDATE\_CUSTOM](https://cmake.org/cmake/help/latest/variable/CTEST_GIT_UPDATE_CUSTOM.html#variable:CTEST_GIT_UPDATE_CUSTOM) should help checking a specific branch or commit.

i also suggest we transition all remaining extensions from **svn** to **git** , and only support **git**. That would help simplify the system.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 7, 2018, 9:38pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/3 "2018-03-07T21:38:15Z")

</div>

@jcfr thank you for this thorough analysis. But, in a nutshell, this basically means we cannot monitor status of the individual extensions at the moment.

I am fully aware about the extremely limited resources, and unfortunately I have no resources to contribute to solving this issue. So I guess we will wait for better times to revisit this…

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 7, 2018, 9:39pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/4 "2018-03-07T21:39:12Z")

</div>

I will follow up shortly, I have meeting with the CDash core developer in few minutes.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 7, 2018, 9:53pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/5 "2018-03-07T21:53:33Z")

</div>

So, as I was thinking about this, conceptually at least, this looks like a very manageable problem to solve (“this” == automatic notification about failing extensions based on the dashboard status).

A while ago, when I ran into this, I wrote [this script](https://github.com/fedorov/CDashWrangler) to parse the dashboard HTML. I would not be surprised if this is broken by now, but I am also confident this can be done for the current incarnation of the the dashboard.

If we forget about the peculiarities of CDash/current system, isn’t it as simple as having a cron job running that kind of script and sending emails? Am I missing something?

What if we have the following setup:

1. upon successful completion of all builds on the dashboard, some webhook is called to initiate a CI build (I think this webhook is the only piece missing).
2. CI script downloads the dashboard HTML page, parses it, and figures out the status of all extensions.
3. For projects hosted on GitHub we can [automatically get the list of contributors](https://developer.github.com/v3/repos/#list-contributors), and send them automatic notifications if dashboard problems with their extension are identified.
4. To keep things simple, we could hard-code the list of extensions for which we would send notifications to not bother with the subscribe/unsubscribe functionality.

---

<div class="post-metadata">

**Author:** ![ihnorton](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/ihnorton/32/9_2.png) [@ihnorton](https://discourse.slicer.org/u/ihnorton)\
**Post date:** [March 7, 2018, 10:21pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/6 "2018-03-07T22:21:46Z")

</div>

Instead of parsing html, you could use the [API](https://www.paraview.org/Wiki/CDash:API).

e.g.: [http://slicer.cdash.org/api/v1/index.php?project=SlicerPreview&filtercombine=or&filtercount=1&showfilters=1&field1=label&compare1=63&value1=radiomics](http://slicer.cdash.org/api/v1/index.php?project=SlicerPreview&filtercombine=or&filtercount=1&showfilters=1&field1=label&compare1=63&value1=radiomics) (for corresponding [html page](http://slicer.cdash.org/index.php?project=SlicerPreview&filtercombine=or&filtercount=1&showfilters=1&field1=label&compare1=63&value1=radiomics))

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [March 8, 2018, 2:05am UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/7 "2018-03-08T02:05:17Z")

</div>

CDash almost does everything that we need and would offer more then the proposed custom scraping/query-based solution (can take into account repository modifications, has convenient web GUI for notification configuration, etc).

Setting up a notification system once may be less work than improving CDash, but in the long term it would probably cost more time to continuously maintain and support a custom system.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 8, 2018, 3:58am UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/8 "2018-03-08T03:58:13Z")

</div>

Indeed, but in absence of a perfect solution, there is no notifications mechanism for years now, and it happens again and again (at least for me!) that extensions get broken and that remains unnoticed.

The good part is that the solution I proposed, in its most simple implementation using a locally running cron job, has 0 dependencies and is easy to implement with minimum resources (unless I am missing something, which may well be). I will update this thread when I get around to do it.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 8, 2018, 3:59am UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/9 "2018-03-08T03:59:30Z")

</div>

> [@ihnorton](#):
>
> Instead of parsing html, you could use the API.

Yes, JC mentioned that in the past as well, but I had troubles locating documentation. I agree, I should search again.

---

<div class="post-metadata">

**Author:** ![vkt1414](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/vkt1414/32/68124_2.png) [@vkt1414](https://discourse.slicer.org/u/vkt1414)\
**Post date:** [November 2, 2023, 10:26am UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/10 "2023-11-02T10:26:52Z")

</div>

thanks to @fedorov and @ihnorton, we have now set up simple GitHub actions to check the status of the extension’s build.

[A github actions workflow to notify if building the extension fails by vkt1414 · Pull Request #16 · ImagingDataCommons/SlicerIDCBrowser](https://github.com/ImagingDataCommons/SlicerIDCBrowser/pull/16)

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [November 2, 2023, 12:52pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/11 "2023-11-02T12:52:16Z")

</div>

Thank you @vkt1414 - this addresses the immediate problem for me! Most definitely, the proper way to address this would be via some kind of a notification or status reporting mechanism from CDash, but clearly it is either not easy, not a priority, or both.

We should go ahead and implement this for all of the extensions we maintain! 👍 🚀

---

<div class="post-metadata">

**Author:** ![vkt1414](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/vkt1414/32/68124_2.png) [@vkt1414](https://discourse.slicer.org/u/vkt1414)\
**Post date:** [November 2, 2023, 1:06pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/12 "2023-11-02T13:06:56Z")

</div>

Great. Github actions is turning out to be an excellent tool to do all the chores.  
any other extensions to keep track of, than below?  
DCMQI  
QuantitativeReporting  
QIICR SDT  
SlicerMRunner (mhub)  
mpReview

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [November 2, 2023, 1:54pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/13 "2023-11-02T13:54:42Z")

</div>

@vkt1414 can you provide some more documentation about this feature? Specifically where does it post the results and how do people set it up for their own extensions?

---

<div class="post-metadata">

**Author:** ![vkt1414](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/vkt1414/32/68124_2.png) [@vkt1414](https://discourse.slicer.org/u/vkt1414)\
**Post date:** [November 2, 2023, 2:31pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/14 "2023-11-02T14:31:58Z")

</div>

Hi @pieper !

Absolutely!

I wrote a simple GitHub actions script that runs a python script that calls the cdash API to get the build info. It extracts the error counts from all categories (configuration, compilation, and test). If the sum of errors is greater than zero, I simply triggered an exit code 1 to fail the GitHub actions, and it should then send a notification.

To adapt this workflow to any extension, copy the .github/ folder to any repo, and then just need to modify the variable ‘slicerExtensionName’ in the get\_build\_status.py, and that’s it. Happy to submit a pull request if you would like me to submit one for any extension!

[SlicerIDCBrowser/.github at main · ImagingDataCommons/SlicerIDCBrowser](https://github.com/ImagingDataCommons/SlicerIDCBrowser/tree/main/.github)

We are not posting the results anywhere, the script just prints the counts of errors to the console.

One big picture idea I had was, if we want to implement for all extensions as part of slicer, we can use an action like this [Send email · Actions · GitHub Marketplace](https://github.com/marketplace/actions/send-email) to send notifications to only developers of the extension, as I believe we can get contributors email addresses from github api. We will need to setup some official slicer email address I guess. Just my two cents.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [November 2, 2023, 2:47pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/15 "2023-11-02T14:47:15Z")

</div>

This is a great idea 💯

What do you think of generalizing this to all extensions ?

Possible workflow:

- When errors (or warnings) are reported for a specific extension, have an issue titled `ExtensionName [Preview]` or `ExtensionName [Stable X.Y]` created in the [Slicer/ExtensionsIndex](https://github.com/Slicer/ExtensionsIndex) GitHub project.
- In the issue description, include a comment to notify code owners associated with this extension
- If an issue for a given extension already exists, it could be closed if there are no more errors.

Some pointers:

- add a workflow to [https://github.com/Slicer/ExtensionsIndex/](https://github.com/Slicer/ExtensionsIndex/)
- add a [CODEOWNERS](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners) file to the `ExtensionsIndex` associating each `.s4ext` with relevant developers
- issue automatically created would be opened by the `slicerbot` GitHub user

Related actions:

- [https://github.com/marketplace/actions/codeowners-action](https://github.com/marketplace/actions/codeowners-action)

---

<div class="post-metadata">

**Author:** ![vkt1414](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/vkt1414/32/68124_2.png) [@vkt1414](https://discourse.slicer.org/u/vkt1414)\
**Post date:** [November 2, 2023, 3:16pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/16 "2023-11-02T15:16:28Z")

</div>

Sounds like a great idea! I have not used the CODEOWNERS file before but the only gripe I have is, that someone will have to maintain the CODEOWNERS file or you make it a requirement, whenever a new extension is added I guess.

I’ll be happy to work on this and report back!

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [November 2, 2023, 3:30pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/17 "2023-11-02T15:30:29Z")

</div>

Sounds great @vkt1414 👍 This should be really helpful.

---

<div class="post-metadata">

**Author:** ![vkt1414](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/vkt1414/32/68124_2.png) [@vkt1414](https://discourse.slicer.org/u/vkt1414)\
**Post date:** [November 13, 2023, 7:25pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/18 "2023-11-13T19:25:38Z")

</div>

Hi @jcfr, @pieper, and @fedorov,

I’ve worked on this and summarized my progress in the attached figure.

 ![Copy of Decision Tree Diagram by Slidesgo (2)](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/1/0/10f76c2f9e8585a0c6dff905bcd7cff600003aad.png)

I’ve been conducting my experiments on a fork, without actually tagging anyone (for now). You can view the issues that GitHub actions bot has created/closed here [link](https://github.com/vkt1414/ExtensionsIndex/issues).

The GitHub Actions workflow I’ve been using can be found [here](https://github.com/vkt1414/ExtensionsIndex/blob/main/.github/workflows/notify_code_owners.yaml), and the CODEOWNERS file is located [here](https://github.com/vkt1414/ExtensionsIndex/blob/main/CODEOWNERS).

I would appreciate any feedback or suggestions for improvement. One area that could use refinement is the CODEOWNERS file. Currently, I check contributors and select the individual who merged the most recent pull request. If more people should be notified, we can edit the file to include them.

This workflow does not interfere if an extension already has a point of contact. However, it will automatically check for new extensions that do not have a CODEOWNER. If you wish to edit it, feel free to do so once I submit the PR. Whether you choose to commit the updated CODEOWNERS back to the repo when an extension lacks a CODEOWNER is entirely up to you. I can add another step to the workflow to accommodate this.

Another thing I would like to know is if there is a github issue for an extension already, I do not update it. Should we? I do not if some extensions are maintained anymore, so we will have a long thread if errors or warnings are not addressed promptly.

Please note that a limitation of this workflow is that non-GitHub repositories cannot be notified.

Looking forward to your feedback.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [November 13, 2023, 7:59pm UTC](https://discourse.slicer.org/t/how-can-i-subscribe-to-monitor-individual-extensions-on-slicer-dashboard/2261/19 "2023-11-13T19:59:07Z")

</div>

@vkt1414 I think this is excellent! Having all issues opened under ExtensionsIndex is very good from the user perspective, I think, since it helps with having a single point to check the status.

If we had a vote, I would definitely support adding this to ExtensionIndex - it’s such a huge improvement over what we have right now (which isn’t much!) in terms of helping developers monitor the dashboard, and informing users about the de facto status of the extensions based on nightly tests.