# CircleCI build failures

**URL:** <https://discourse.slicer.org/t/circleci-build-failures/20828>\
**Category:** Development\
**Created:** [November 29, 2021, 2:36pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828 "2021-11-29T14:36:10Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 29, 2021, 2:36pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/1 "2021-11-29T14:36:10Z")

</div>

A few weeks ago all our circleCI builds on pull requests (such as [this](https://app.circleci.com/pipelines/github/Slicer/Slicer/2796/workflows/90ce1a63-466e-4a1f-acde-9ed2b8223896/jobs/2796)) started to fail because the build is not completed within 60 minutes.

I’ve checked our usage and it seems that we would need to pay at least $100 per month ([CircleCI](https://app.circleci.com/settings/plan/github/Slicer/overview?intent=purchase-performance-plan)) to cover our needs, and I’m not sure if the maximum build timeout can be adjusted even then. If it can be then we may potentially need to pay more. Another risk of the “pay-as-you-go” pricing of circleCI is that if we mess up something and builds don’t end for many hours and we don’t notice that in time then the charges can go up a lot.

Should we try to speed up the build? For example we could try creating the docker image after Slicer build is completed and do an incremental build instead of a full build for each pull request. This would potentially miss some rare build errors (that only occur when building from scratch), but we would get results much faster, so developers could fix the issues more quickly and circleCI would charge less (or could remain free).

Or, should we switch to GitHub actions?

Or set up runners on our computers? It would be no problem to allocate a few computers in the PerkLab for continuous builds, if there is someone to help me with the setup.

@jcfr @Sam_Horvath @pieper

---

<div class="post-metadata">

**Author:** ![adamrankin](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/adamrankin/32/155_2.png) [@adamrankin](https://discourse.slicer.org/u/adamrankin)\
**Post date:** [November 29, 2021, 2:42pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/2 "2021-11-29T14:42:59Z")

</div>

We (VASST, Robarts) can also contribute some runners if that’s the route we take.

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [November 29, 2021, 3:01pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/3 "2021-11-29T15:01:31Z")

</div>

I’d be happy to contribute runner machines, but in general we’d get a lot more value from breaking up the build into manageable sets of dependencies that could be rebuilt intelligently. I think gh actions could be a better infrastructure for this since we can set up a build matrix. But it will require someone working through all the details. Runners might be an easier short-term solution.

---

<div class="post-metadata">

**Author:** ![Sam\_Horvath](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/sam_horvath/32/3092_2.png) [@Sam\_Horvath](https://discourse.slicer.org/u/Sam_Horvath)\
**Post date:** [November 29, 2021, 3:59pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/4 "2021-11-29T15:59:46Z")

</div>

We will look into why the build is going over time. For the CircleCI builds we are using a Docker image with the dependencies already built.

We can look at doing an incremental build of Slicer for CircleCI

---

<div class="post-metadata">

**Author:** ![Sam\_Horvath](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/sam_horvath/32/3092_2.png) [@Sam\_Horvath](https://discourse.slicer.org/u/Sam_Horvath)\
**Post date:** [November 29, 2021, 4:06pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/5 "2021-11-29T16:06:23Z")

</div>

This appears to be the first commit where the timeout failure becomes permanent: [BUG: Fix window/level presets in Volumes module · Slicer/Slicer@409c7f0 · GitHub](https://github.com/Slicer/Slicer/commit/409c7f035aa9015863cffc827ed37661322a550d)

However, looking at older commits that passed, it seems that the timeout has changed? Older commits are going as much as 2 hrs in some cases and [passing](https://app.circleci.com/pipelines/github/Slicer/Slicer/2693/workflows/ca7cd642-7301-44cc-be22-dfd7b9972db3/jobs/2693). We would need to trim quite a bit off the build to fix that it seems, so an incremental build may be necessary.

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [November 29, 2021, 4:14pm UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/6 "2021-11-29T16:14:57Z")

</div>

> [@Sam\_Horvath](#):
>
> However, looking at older commits that passed, it seems that the timeout has changed?

Yes, it seems that circleCI has changed its policy. I saw that a few weeks ago ITK maintainers disabled circleCI builds, too, for the same reason.

---

<div class="post-metadata">

**Author:** ![jcfr](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jcfr/32/17825_2.png) [@jcfr](https://discourse.slicer.org/u/jcfr)\
**Post date:** [November 30, 2021, 6:17am UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/7 "2021-11-30T06:17:22Z")

</div>

This should be addressed by the following pull request:

> <https://github.com/Slicer/Slicer/pull/6051>
>
> To avoid the newly introduced 60-minute CircleCI limit, this commit
> removes the… continuous integration pipeline based on CircleCI and adds
> a new workflow based on GitHub Action.
> 
> The new CI workflow also:
> \- introduces a "private" action called "slicer-build" responsible for
> fetching the latest "slicer/slicer-base" image and starting the build
> \- uploads the generated Slicer package as an artifact with a retention
> time of 1 day.
> 
> See https://discourse.slicer.org/t/circleci-build-failures/20828

---

<div class="post-metadata">

**Author:** ![rbumm](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/rbumm/32/9404_2.png) [@rbumm](https://discourse.slicer.org/u/rbumm)\
**Post date:** [December 1, 2021, 8:01am UTC](https://discourse.slicer.org/t/circleci-build-failures/20828/8 "2021-12-01T08:01:12Z")

</div>

It would be great to have a faster initial build after a new Slicer fork / branch.

What does make the S4R or S4D directory so extremely big?  
Windows: Could we not just provide the outer build files (DLLs and lib files) as they are and let them only compile new if dependencies change?