# How to generate a test coverage report for scripted module?

**URL:** <https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714>\
**Category:** Development\
**Tags:** testing\
**Created:** [November 21, 2021, 5:41pm UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714 "2021-11-21T17:41:04Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![pll\_llq](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pll_llq/32/9408_2.png) [@pll\_llq](https://discourse.slicer.org/u/pll_llq)\
**Post date:** [November 21, 2021, 5:41pm UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/1 "2021-11-21T17:41:04Z")

</div>

The title says it all.

Is there a way to generate a test coverage report based on the contents of the `ScriptedLoadableModuleTest` class contents in a scripted module?

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [November 22, 2021, 6:13pm UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/2 "2021-11-22T18:13:47Z")

</div>

I’m not aware that anyone has tried that, but if I’m not mistaken the scripted module tests are simply pytest invoked from cmake. So if that’s right you could probably use the pytest coverage tools. If you come up with a recipe be sure to share it back.

---

<div class="post-metadata">

**Author:** ![Alex\_Vergara](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/alex_vergara/32/15205_2.png) [@Alex\_Vergara](https://discourse.slicer.org/u/Alex_Vergara)\
**Post date:** [November 23, 2021, 9:32am UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/3 "2021-11-23T09:32:50Z")

</div>

I have tried that, it doesn’t work. See [https://gitlab.com/opendose/opendose3d/-/blob/develop/.gitlab-ci.yml](https://gitlab.com/opendose/opendose3d/-/blob/develop/.gitlab-ci.yml)

The relevant pytest lines are commented out because they prevent the module to be built inside CDASH

---

<div class="post-metadata">

**Author:** ![pll\_llq](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pll_llq/32/9408_2.png) [@pll\_llq](https://discourse.slicer.org/u/pll_llq)\
**Post date:** [November 23, 2021, 10:05am UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/4 "2021-11-23T10:05:28Z")

</div>

Thanks for the info. I’ll look into what `coverage` package can provide and if there can be an easy hack to make it work

---

<div class="post-metadata">

**Author:** ![pll\_llq](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pll_llq/32/9408_2.png) [@pll\_llq](https://discourse.slicer.org/u/pll_llq)\
**Post date:** [November 24, 2021, 1:27pm UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/5 "2021-11-24T13:27:24Z")

</div>

Thanks everyone🖖

Yesterday at the developer call @jcfr steered me towards looking into the API of the `coverage` package and trying to run execution through the Slicer environment.

The `coverage` package is _pip\_installable_ into Slicer and has a pretty simple API.  
I came up with this python script that can generate `selfTests` coverage report for a scriptable module:

```python
import os
from coverage import Coverage

# Setup test execution recording. An sqlite database file will be created here
cov = Coverage(data_file=PATH_TO_COVERAGE_DB)

# Record what lines of code are executed during self test run
cov.start()
slicer.selfTests[MY_MODULE_NAME]()
cov.stop()

# List python files that should be mentioned in the report
extensionPythonFiles = []
for root, dirNames, fileNames in os.walk(PATH_TO_MY_MODULE_FOLDER):
    fileNames = [f for f in fileNames if f.endswith('.py') and not f.startswith('__')]
    for f in fileNames:
        extensionPythonFiles.append(os.path.join(root, f))

# Generate report to a text file
with open(PATH_TO_REPORT_FILE, 'w') as rf:
    cov.report(morfs=extensionPythonFiles, file=rf, show_missing=True)

```

---

<div class="post-metadata">

**Author:** ![Alex\_Vergara](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/alex_vergara/32/15205_2.png) [@Alex\_Vergara](https://discourse.slicer.org/u/Alex_Vergara)\
**Post date:** [November 26, 2021, 8:04am UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/6 "2021-11-26T08:04:37Z")

</div>

I think this doesn’t solve the problem. Although it generates the report inside slicer itself, it does not generate the report on build. The build is made using cmake and the tests are done through ctest. This last procedure is not generating reports.

---

<div class="post-metadata">

**Author:** ![pll\_llq](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pll_llq/32/9408_2.png) [@pll\_llq](https://discourse.slicer.org/u/pll_llq)\
**Post date:** [November 26, 2021, 8:13am UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/7 "2021-11-26T08:13:13Z")

</div>

I might have not described my use case. At the moment I work only with scriptable modules that I run against the latest slicer so I am not doing builds on the CI. I wanted to have a look at my coverage on the python side and I didn’t look at how to integrate this into the build process.

---

<div class="post-metadata">

**Author:** ![Alex\_Vergara](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/alex_vergara/32/15205_2.png) [@Alex\_Vergara](https://discourse.slicer.org/u/Alex_Vergara)\
**Post date:** [November 26, 2021, 8:15am UTC](https://discourse.slicer.org/t/how-to-generate-a-test-coverage-report-for-scripted-module/20714/8 "2021-11-26T08:15:44Z")

</div>

for now I am creating the reports as txt inside slicer, upload them in git pull inside my repo and loading the report after the ctest, UGLY but is the only thing that’s working for me ATM