Thanks for the report 
Looking at CDash associated with both the Stable (see here) and the Preview build (see here), the extension is packaged and available on Linux and macOS but not on Windows.
For example, corresponding Windows error can be reviewed here
A dedicated issue has been created: https://github.com/Slicer/SlicerJupyter/issues/78
@Matteboo Let us know if you would like to either work on the issue or if you would have resources to help support its resolution