# ExtensionIndex 4.8 branch is using master for many (most?) extensions

**URL:** <https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222>\
**Category:** Development\
**Tags:** extensions-manager\
**Created:** [March 1, 2018, 5:35pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222 "2018-03-01T17:35:58Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 5:35pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/1 "2018-03-01T17:35:58Z")

</div>

I noticed that many of the extensions in the ExtensionsIndex `4.8` branch are using `master` for the extension repo. I initially thought I missed this, but it applies to many popular extensions. This is a big issue considering the breaking changes in Slicer post-4.8.

I am assuming the 4.8 branch was just forked from the master of ExtensionsIndex? Should we establish a process to automatically update all references to `master` to the latest hash at the time ExtensionsIndex release branch is forked, or at least ask the extensions developers to update the index files?

Problem is, right now it is not straightforward to find the latest hash that was working and was tested against 4.8 branch of Slicer.

8-/

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [March 1, 2018, 6:03pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/2 "2018-03-01T18:03:50Z")

</div>

For the extensions that we maintain, we try to keep them compatible with both 4.8 and master as long as it is possible. It is not by mistake but to reduce maintenance effort: we don’t have to keep applying fixes and improvements to two branches. About half of our extensions still use the same branch for 4.8 and master version of Slicer, but as there are more and more differences, probably eventually all of them will have two branches.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 6:08pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/3 "2018-03-01T18:08:17Z")

</div>

I see. On our side, it was definitely not intentional. We just missed the update in the 4.8. For some of the extensions, switch to the updated DCMTK is breaking due to changes in the API, and in other extensions there are modifications/improvements which are not yet working, and the result is that the extension is broken both in the nightly and latest stable.

I think it is also important for managing user expectations - master by definition is the place for experiments. It is often not possible to test under the conditions of the dashboard before committing to master, and once you committed, and the result is not working, extension will be broken in stable, which is, well, NOT stable.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 1, 2018, 6:47pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/4 "2018-03-01T18:47:00Z")

</div>

> [@fedorov](#):
>
> On our side, it was definitely not intentional. We just missed the update in the 4.8.

Choosing to use `master` branch instead of explicitly updating the hash implies great responsibilities, that is why this is not the default.

> Problem is, right now it is not straightforward to find the latest hash that was working and was tested against 4.8 branch of Slicer.

Exactly, using `master` branch prevents from doing any forensic and understanding when things break.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 7:18pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/5 "2018-03-01T19:18:22Z")

</div>

No question - we screwed up big time, we are to blame, and we have to fix it.

But still, automatic replacement of all references to master with a current (at the time release is cut) master hash would be relatively easy to implement, and would probably improve user experience.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 1, 2018, 7:19pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/6 "2018-03-01T19:19:52Z")

</div>

> [@fedorov](#):
>
> automatic replacement of all references to master with a current (at the time release is cut) master hash would be relatively easy to implement, and would probably improve user experience.

👍 This would be great

---

<div class="post-metadata">

**Author:** ![cpinter](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/cpinter/32/7995_2.png) [@cpinter](https://discourse.slicer.org/u/cpinter)\
**Post date:** [March 1, 2018, 7:21pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/7 "2018-03-01T19:21:49Z")

</div>

I agree with Andrey, updating the extension hashes to the one that was latest at the time of releasing Slicer stable would be an easy solution for this. For extensions that are actively maintained, this hash fixation would make sure that the extension works for the stable, which they can update at their own leisure later. For extensions that are broken… they will remain broken either way.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 8:07pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/8 "2018-03-01T20:07:37Z")

</div>

I am glad we are in agreement!

I will take the initial pass on the script and make the PR to the ExtensionsIndex repo.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 1, 2018, 8:07pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/9 "2018-03-01T20:07:58Z")

</div>

To summarize, the release checklist could include an extra step:

- Trigger build for Slicer release
- Next day, on one of the factory, run script `updateExtensionIndex` in extension index build directory.

Pseudo code for `updateExtensionIndex` script:

```
Input parameters are:
    extensionindex_build_dir .. : Contains a checkout of the extension index
                                                 as well as all source and build dir of each extensions
    release ...................: Version to associate with the next extension index release branch

(0) Set extensonindex_dir based on extensionindex_build_dir

(1) Get ${extensionName} associated with all ${extensonindex_dir}/*.s4ext

(2) for each ${extensionName}
        cd ${extensionName}
        latest_scmrevision=$(git rev-list -n1 HEAD)
        sed -e "s/scmrevision.*/scmrevision ${latest_scmrevision}/" -i ${extensionName}.s4ext

(3) cd ${extensonindex_dir}

(4) git checkout -b master-${release}

(5) git add -A

(6) git commit -m "Ensure all description files references a specific revision" # done by slicerbot user

(7) git push origin master-${release}
```

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 1, 2018, 8:09pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/10 "2018-03-01T20:09:29Z")

</div>

Assumptions for the script would be:

- all extension source were successfully checked out and are using `git`

- extension maintainer will be responsible to update the file to use `master-X.Z` or similar afterward

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 9:06pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/11 "2018-03-01T21:06:15Z")

</div>

PR for discussion/testing etc: [https://github.com/Slicer/ExtensionsIndex/pull/1529](https://github.com/Slicer/ExtensionsIndex/pull/1529)

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 10:19pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/12 "2018-03-01T22:19:26Z")

</div>

@jcfr the assumptions you make are not compatible with what I had in mind.

> [@jcfr](#):
>
> all extension source were successfully checked out and are using git

What I had in mind is a script that does not assume that all extensions are checked out, and leave it to the person cutting the release to run it after the branch is created.

> [@jcfr](#):
>
> extension maintainer will be responsible to update the file to use master-X.Z or similar afterward

I think at the time release is prepared, the authority should be with the release maintainer. The release process should not be interrupted by lack of response from the extension maintainer. I also think that it is the courtesy to the future user to “freeze” the functionality of the extension as the default behavior as it was at the time of the release. Advanced extension developers can always revert back.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [March 1, 2018, 11:12pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/13 "2018-03-01T23:12:29Z")

</div>

> leave it to the person cutting the release to run it after the branch is created.

Makes sense.

When the script will be finalized and tested (e.g on a fork of the extension index), I suggest we update [this step](https://www.slicer.org/wiki/Documentation/Nightly/Developers/ReleaseProcess#Update_ExtensionsIndex) of the release process

> [@fedorov](#):
>
> I think at the time release is prepared, the authority should be with the release maintainer. […]

👍

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 1, 2018, 11:26pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/14 "2018-03-01T23:26:16Z")

</div>

Great, thanks! I will revise the script to add some documentation comments, will test with a fork, and will update the issue then.

Also, as I was going through this process, I discovered that some extension use `scmversion release` which is a misnomer considering Slicer release process. We should probably follow up with issues for those extensions to resolve this ambiguity…

```auto
fedorov@radiobeat [18:22:14] [~/github/ExtensionsIndex] [master]
% grep release *
DTI-Reg.s4ext:scmrevision release
DTIAtlasBuilder.s4ext:scmrevision release
DTIAtlasFiberAnalyzer.s4ext:scmrevision release
DTIPrep.s4ext:scmrevision release
DTIProcess.s4ext:scmrevision release
DatabaseInteractor.s4ext:scmrevision release
SPHARM-PDM.s4ext:scmrevision release
ShapePopulationViewer.s4ext:scmrevision release
ShapeVariationAnalyzer.s4ext:scmrevision release

```

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [March 8, 2018, 3:51am UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/15 "2018-03-08T03:51:22Z")

</div>

I updated the PR, it is ready for review now. Once merged, I will update the wiki instructions.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [November 16, 2018, 9:31pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/16 "2018-11-16T21:31:47Z")

</div>

@lassoan @jcfr

I just realized [the PR I proposed to help addressing this issue](https://github.com/Slicer/ExtensionsIndex/pull/1529) was never reviewed/acted upon, and we’ve just had another release where extensions in the release branch are still using “master” for their versions (e.g., see QuantitativeReporting in 4.10 [here](https://github.com/Slicer/ExtensionsIndex/blob/4.10/QuantitativeReporting.s4ext#L10)).

Given we reached consensus back in Spring, can you consider this PR?

Whatever is the decision on the PR, we should definitely fix references to master in the extensions included in [https://github.com/Slicer/ExtensionsIndex/tree/4.10](https://github.com/Slicer/ExtensionsIndex/tree/4.10)!

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 16, 2018, 10:39pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/17 "2018-11-16T22:39:22Z")

</div>

Projects that are not maintained anymore will not get new revisions and therefore remain compatible with latest stable. Actively maintained projects are assumed to maintain their ExtensionsIndex entries, too. When versions diverge, then typically separate branch is created (hash is not used even then).

[Custom Slicer applications](https://github.com/KitwareMedical/SlicerCustomAppTemplate) do not use the ExtensionsIndex but all extension versions are listed in the main application repository.

Pinning specific git hashes would only help developers, who use branch name in ExtensionsIndex, maintain their extensions, introduce backward incompatible changes, and forget about creating a separate stable branch in their extension repository. This is probably affects just a few extensions.

Anyway, I think it may still worth pinning extension versions using hashes, as it seems simple to do and I don’t see any disadvantages.

---

<div class="post-metadata">

**Author:** ![fedorov](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/fedorov/32/14_2.png) [@fedorov](https://discourse.slicer.org/u/fedorov)\
**Post date:** [November 16, 2018, 11:15pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/18 "2018-11-16T23:15:15Z")

</div>

> [@lassoan](#):
>
> Projects that are not maintained anymore will not get new revisions and therefore remain compatible with latest stable.

Except when there are changes that are not backwards compatible, which did happen for us as I discussed in one of the earlier posts (backwards incompatible API change in DCMTK).

> [@lassoan](#):
>
> Actively maintained projects are assumed to maintain their ExtensionsIndex entries, too.

I would not assume that. People forget. I did forget about the need to freeze hash or make a branch once the release was cut, and I don’t think there was any reminder sent out. But hey - I may well be an outlier! 🙂 It may well be other developers have longer memory.

> [@lassoan](#):
>
> This is probably affects just a few extensions.

I do not have any data to support or refute that statement.

Anyway, enough trying to “save the world”! For the sake of expediency, I am just going to update the extensions I maintain, and send reminders to the developers for the extensions I collaborated on.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 16, 2018, 11:40pm UTC](https://discourse.slicer.org/t/extensionindex-4-8-branch-is-using-master-for-many-most-extensions/2222/19 "2018-11-16T23:40:21Z")

</div>

> [@fedorov](#):
>
> Except when there are changes that are not backwards compatible, which did happen for us

How it is possible? If you don’t make any incompatible change in your extension then it will not break. You can keep using the same branch for both master and stable, and users will keep getting updates. If course things can change in the master branch, which may force you to make changes, but you may still manage that in the same code (e.g., using version checks/ifdefs).

If you are not sure if a change is backward-compatible and don’t have the capacity to test it, then you may decide to stop maintaining the stable branch. I would still not freeze it with a hash but instead create a stable branch in your repository and writing that branch name in the ExtensionsIndex. It would be very clear in your repository (much more explicit than some git hash in another repository) and you could also backport any fixes easily into this branch.