# Introduction of "tiers" for Slicer extensions

**URL:** <https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870>\
**Category:** Development\
**Tags:** extensions-manager\
**Created:** [March 13, 2024, 4:56pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870 "2024-03-13T16:56:24Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [March 13, 2024, 4:56pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/1 "2024-03-13T16:56:24Z")

</div>

_The definition of tiers was revisited during July 16th weekly meeting\[1\] and this post was updated accordingly._

During our last Slicer developer hangout of March 13th 2024, we discussed the issue that review and approval of new extensions often takes very long time. We identified that the main difficulty was that extensions had to meet many requirements to be admitted to the Extensions Catalog. Reducing the requirements could make extensions approval easier but it could result in having lower quality extensions (harder to use, not well documented or tested, etc.).

To address this problem, we propose organization of Slicer extensions into “tiers”. This helps users understand how mature, well tested, and well supported an extension is and allows extension developers to choose the amount of effort that they are comfortable with investing into the extension.

By default, the Extensions Manager would only show extensions that are in Tier \>=3 and users would need to click somewhere in the Extensions Manager to see all extensions. This allows new users to more easily find relevant extensions and less likely to come across experimental extensions that may be hard to use or not well documented.

The proposed tiers are the following:

## Tier 5

Critically important extensions, supported by Slicer core developers.

- Slicer core developers accept the responsibility of fixing any issues caused by Slicer core changes; at least one Slicer core developer (anyone who has commit right to Slicer core) must be granted commit right to the extension’s repository
- Automated tests for all critical features
- Maintainers respond to questions related to the extension on the [Slicer Forum](https://discourse.slicer.org)
- All requirements of tiers \< 5

## Tier 3

Community-supported extensions.

- Documentation, tutorial, and test data are provided for most modules. A tutorial provides step-by-step description of at least the most typical use case, include a few screenshots. Any sample data sets that is used in tutorials must be registered with the Sample Data module to provide easy access to the user.
- Follows programming and user interface conventions of 3D Slicer (e.g., GUI and logic are separated, usage of popups is minimized, no unnecessary custom GUI styling, etc.)
- Maintainers respond to issues and pull request submitted to the extension’s repository
- Maintainers respond to questions directly addressed to him/her via @mention on the [Slicer Forum](https://discourse.slicer.org)
- Permissive license is used for the main functions of the extension (recommended Apache or MIT). The extension can provide additional functionality in optional components that are distributed with non-permissive license, but the user has to explicitly approve those before using them (e.g., a pop-up can be displayed that explains the licensing terms and the user has to acknowledge them to proceed).
- All requirements of tiers \< 3

## Tier 1

Experimental extensions.

- Extension has a reasonable name (not too general, not too narrow, suggests what the extension is for)
- Repository name is Slicer+ExtensionName
- Repository is associated with `3d-slicer-extension` GitHub topic so that it is listed [here](https://github.com/topics/3d-slicer-extension). To edit topics, click the settings icon in the right side of “About” section header and enter `3d-slicer-extension` in “Topics” and click “Save changes”. To learn more about topics, read [Classifying your repository with topics - GitHub Docs](https://help.github.com/en/articles/about-topics)
- Extension description summarizes in 1-2 sentences what the extension is usable (should be understandable for non-experts)
- Any known related patents must be mentioned in the extension description.
- LICENSE.txt is present in the repository root. MIT ([MIT License | Choose a License](https://choosealicense.com/licenses/mit/)) or Apache ([Apache License 2.0 | Choose a License](https://choosealicense.com/licenses/apache-2.0/)) license is recommended. If source code license is more restrictive for users than MIT, BSD, Apache, or 3D Slicer license then the name of the used license must be mentioned in the extension description.
- Extension URL and revision (scmurl, scmrevision) is correct, consider using a branch name (main, release, …) instead of a specific git hash to avoid re-submitting pull request whenever the extension is updated
- Extension icon URL is correct (do not use the icon’s webpage but the raw data download URL that you get from the download button - it should look something like this: [https://raw.githubusercontent.com/user/repo/main/SomeIcon.png](https://raw.githubusercontent.com/user/repo/main/SomeIcon.png))
- Screenshot URLs (screenshoturls) are correct, contains at least one
- Homepage URL points to valid webpage containing the following:
  - Extension name
  - Short description: 1-2 sentences, which summarizes what the extension is usable for
  - At least one nice, informative image, that illustrates what the extension can do. It may be a screenshot.
  - Description of contained modules: at one sentence for each module
  - Publication: link to publication and/or to PubMed reference (if available)
  - License: We suggest you use a permissive license that includes patent and contribution clauses. This will help protect developers and ensure the code remains freely available. We suggest you use the [Slicer License](https://github.com/Slicer/Slicer/blob/main/License.txt) or the [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0). Always mention in your README file the license you have chosen. If you choose a different license, explain why to the extension maintainers. Depending on the license we may not be able to host your work. Read [here](https://opensource.guide/legal/#which-open-source-license-is-appropriate-for-my-project) to learn more about licenses.

- Hide unused github features (such as Wiki, Projects, and Discussions, Releases, Packages) in the repository to reduce noise/irrelevant information:
  - Click `Settings` and in repository settings uncheck `Wiki`, `Projects`, and `Discussions` (if they are currently not used).
  - Click the settings icon next to `About` in the top-right corner of the repository main page and uncheck `Releases` and `Packages` (if they are currently not used)

- The extension is safe
  - Does not include or download binaries from unreliable sources
  - Does not send any information anywhere without user consent (explicit opt-in is required)

* * *

Extension tier 2 and 4 are reserved for future use, to allow defining intermediate steps between the initially defined tiers without reclassifying every extension.

## Extensions catalog entry file

We also propose to switch from the “Extension description (.s4ext)” file format to a simpler, standard JSON format that only contain necessary information. This file will be called `Extensions Catalog entry` json file. The filename is the exension’s name (e.g., SlicerOpenIGTLink.json) and it contains the following content:

```auto
{
    "scmurl": "https://github.com/openigtlink/SlicerOpenIGTLink.git",
    "scmrevision": "master",
    "depends": [],
    "category": "IGT"
    "maintainers": ["https://discourse.slicer.org/u/lassoan", "https://discourse.slicer.org/u/sunderlandkyl"]
    "tier": 5
}

```

* * *

Any comments and suggestions are welcome.

@pieper @jcfr @Sam_Horvath @cpinter @jamesobutler @Sunderlandkyl @RafaelPalomar @mau_igna_06 @muratmaga @rbumm @fedorov

* * *

1. [2024.07.16 Weekly Meeting](https://discourse.slicer.org/t/2024-07-16-weekly-meeting/37386)

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [March 13, 2024, 5:29pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/2 "2024-03-13T17:29:00Z")

</div>

This is great.

It will help to filter the extension manager for older or unsupported extensions.  
Concerning the opt-in for sending information from extensions it would be nice to have demo source code available including the don’t ask again feature, maybe in the script repo.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 13, 2024, 7:07pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/3 "2024-03-13T19:07:11Z")

</div>

For reference, corresponding changes are being implemented in the following pull requests:

- [Switch extension index entry format from "s4ext" to "json" by jcfr · Pull Request #7629 · Slicer/Slicer · GitHub](https://github.com/Slicer/Slicer/pull/7629)
- [CI: Transition from s4ext to json by jcfr · Pull Request #2011 · Slicer/ExtensionsIndex · GitHub](https://github.com/Slicer/ExtensionsIndex/pull/2011)

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 13, 2024, 7:10pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/4 "2024-03-13T19:10:20Z")

</div>

> [@lassoan](#):
>
> `depends`

Since we will be able to decouple built-time from runtime dependencies, I suggest we rename `depends` to `build_depends`

The “depends” metadata currently set in the extension `CMakeLists` would then correspond to the runtime dependencies (aka extension expected to be installed after clicking on the “install” button)

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [March 13, 2024, 7:34pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/5 "2024-03-13T19:34:42Z")

</div>

> [@jcfr](#):
>
> Since we will be able to decouple built-time from runtime dependencies, I suggest we rename `depends` to `build_depends`

Renaming sounds good. I would slightly prefer using full words, such as `build_dependencies`.

In the long term, both build and runtime dependencies should be specified in the repository, in a separate json file so that the both the extension build script can easily read it (to determine the order of build) and CMake files in the extension can access it (so that it can check that all required directories are set, etc.). At that point, the dependencies can be removed from the extensions catalog entry file.

---

<div class="post-metadata">

**Author:** ![chir.set](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/chir.set/32/66982_2.png) [@chir.set](https://discourse.slicer.org/u/chir.set)\
**Post date:** [March 13, 2024, 8:30pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/6 "2024-03-13T20:30:53Z")

</div>

> [@lassoan](#):
>
> we discussed the issue that review and approval of **new extensions** often takes very long time

Does this concern extensions already in the extension repository?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [March 13, 2024, 9:17pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/7 "2024-03-13T21:17:12Z")

</div>

Yes, we’ll need to assign every extension to a tier. I would expect that most extensions that are currently actively maintained will be in Tier 3.

---

<div class="post-metadata">

**Author:** ![jamesobutler](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jamesobutler/32/7511_2.png) [@jamesobutler](https://discourse.slicer.org/u/jamesobutler)\
**Post date:** [May 1, 2024, 5:14pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/8 "2024-05-01T17:14:23Z")

</div>

> [@jcfr](#):
>
> For reference, corresponding changes are being implemented in the following pull requests:
> 
> - [Switch extension index entry format from “s4ext” to “json” by jcfr · Pull Request #7629 · Slicer/Slicer · GitHub](https://github.com/Slicer/Slicer/pull/7629)
> - [CI: Transition from s4ext to json by jcfr · Pull Request #2011 · Slicer/ExtensionsIndex · GitHub](https://github.com/Slicer/ExtensionsIndex/pull/2011)

Will setting tiers for all the existing extensions and changing the extensions review process to be more flexible for lower tier extensions happening soon? It appears that the above mentioned PRs regarding switch to the new json format have been completed.

---

<div class="post-metadata">

**Author:** ![jamesobutler](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jamesobutler/32/7511_2.png) [@jamesobutler](https://discourse.slicer.org/u/jamesobutler)\
**Post date:** [May 1, 2024, 7:05pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/9 "2024-05-01T19:05:50Z")

</div>

> [@lassoan](#):
>
> Maintainers respond to issues and pull request submitted to the extension’s repository  
> Maintainers respond to questions directly addressed to him/her via @mention on the [Slicer Forum](https://discourse.slicer.org)

^ These requirements in Tier 3 probably exclude any new developer or a new extension from starting out being published at Tier 3. This is because there isn’t a record yet if the maintainer will respond to questions on the forum and also unclear if the maintainer responds to issues and PR with a brand new extension that hasn’t been used yet.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [May 2, 2024, 5:11am UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/10 "2024-05-02T05:11:29Z")

</div>

> [@jamesobutler](#):
>
> These requirements in Tier 3 probably exclude any new developer or a new extension from starting out being published at Tier 3.

We could accept a promise from newcomers to be responsive (and downgrade the extension if the developer turns out not to be committed enough).

But I would also find it reasonable to limit accessible tiers for extensions that don’t have any established community member among its developers (and the extension would be upgraded over time if the developer team proves to be responsive).

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [May 2, 2024, 1:34pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/11 "2024-05-02T13:34:47Z")

</div>

> Will setting tiers for all the existing extensions and changing the extensions review process to be more flexible for lower tier extensions happening soon?

Now that we just finalized the update of all [opened pull-requests](https://github.com/Slicer/ExtensionsIndex/pulls?q=is%3Aopen+is%3Apr) so that they include a `json` file instead of a `s4ext` file, this will happen next 🚀

In the meantime, you can also see [here](https://github.com/Slicer/ExtensionsIndex/issues/2041#issuecomment-2088786459) the list of pull requests consolidating extension metadata based on the corresponding s4ext that was contributed.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [July 16, 2024, 6:29pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/12 "2024-07-16T18:29:32Z")

</div>

**Update:** The definition of tiers was revisited during [July 16th weekly meeting](#footnote-108214-1) and this post was updated accordingly.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 25, 2024, 5:52pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/13 "2024-11-25T17:52:28Z")

</div>

A pull request has been submitted that assigns tier value for each extension. By default, tier 3 is used for all extensions - except extensions that are not maintained or has other issues are assigned tier 1, and highly important extensions are assigned tier 5. It is expected that tiers will be finalized within about a week. Please review and if you have any comments (e.g., suggest changing tier of a specific extension) then submit it to the pull request:

> <https://github.com/Slicer/ExtensionsIndex/pull/2123>
>
> By default, Tier 3 is assigned to all extensions - see list of exceptions (that …assigned tier 1 or tier 5 instead) in the commit message.
> 
> Associated schema update: https://github.com/Slicer/Slicer/pull/8066
> 
> Required for https://github.com/Slicer/Slicer/issues/7474

---

<div class="post-metadata">

**Author:** ![mau\_igna\_06](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/mau_igna_06/32/9056_2.png) [@mau\_igna\_06](https://discourse.slicer.org/u/mau_igna_06)\
**Post date:** [November 25, 2024, 9:23pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/14 "2024-11-25T21:23:03Z")

</div>

Just like to put my grain of sand here. For me the extension tiers would mean:  
5 → Updated documentation and works well on stable release  
4 → Old documentation but works well on stable release  
3 → Works well on earlier release and has old documentation  
2 → Works well on earlier release but no documentation  
1 → Does not work

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 25, 2024, 9:57pm UTC](https://discourse.slicer.org/t/introduction-of-tiers-for-slicer-extensions/34870/15 "2024-11-25T21:57:43Z")

</div>

> 1 → Does not work

If an extension does not work then it is archived (extension description is moved to the `ARCHIVE` folder.

> [@mau\_igna\_06](#):
>
> 5 → Updated documentation and works well on stable release  
> 4 → Old documentation but works well on stable release  
> 3 → Works well on earlier release and has old documentation  
> 2 → Works well on earlier release but no documentation

All these will be mapped to the same tier (Tier 3) because it would be very expensive (thousands of dollars) to test all 200+ extensions or check their documentation after each release. Even if we had the funding to do this now, it would not be sustainable and it would not scale. We already have trouble keeping up with just reviewing new extensions. In contrast, we can easily assess available support: we can see when users ask questions or report problems and developers do not respond. If this happens then we can downgrade an extension’s tier. If a developer requests it and the requirements are met then we can raise an extension’s tier.

Tier 5 is distinguished from tier 3 by their maintainers (same people as Slicer core), long term availability, and quality (additional quality requirements in addition to tier 3). These could be independent attributes. However, it is simpler for users if we can map all properties to a single linear scale. We can revisit in the future if we find that is too much of an oversimplification.