With the growing desire to have video compression accessible within Slicer, it would be useful to have a discussion regarding how to go about integrating it into the Slicer core.
Adding a video codec interface into the Slicer core:
Due to licensing issues, it will be unfeasible to include some codecs within the Slicer core. This is particularly sensitive for commercial applications and Linux distros.
Several modules (OpenIGTLinkIF, Sequences, Screen Capture, etc.) also need to have access to the same video compression classes.
It may make sense to create a plugin interface in Slicer to register codecs which can be used by any extension.
Codecs could be downloaded individually through the extension manager, combined together in a codec pack, or registered by installed extensions.
This interface could interfere with the codecs in OpenIGTLink. How should these codecs be synchronized between Slicer and OpenIGTLink?
A related topic is whether we should move OpenIGTLinkIF from the Slicer core to its own extension.
This would allow OpenIGTLinkIF to receive more frequent updates, since we currently have no easy way of updating it for fixes and new features.
This is also affected by the video compression integration, due to the codec licensing issue. If OpenIGTLinkIF was moved to its own extension, then this would be less of an issue.
I think topic 1 is quite clear - we would benefit a lot from having video compression infrastructure in the Slicer core, and having it in extensions would allow us greater flexibility and wider choice of codecs.
OpenIGTLinkIF in core/extension: I don’t see much advantage of having OpenIGTLinkIF in the core, other than slightly increased visibility of the feature (but IGT category name is not that welcoming and if people don’t know what the module is for, they cannot figure it out by opening the module), and there are many small disadvantages (we cannot update it in stable builds, we don’t get specific download statistics, the IGT category is not highlighted when we install IGT extensions, …). It seems that using custom codecs would be easier to set up if OpenIGTLinkIF is in a separate extension (OpenIGTLink library could use the codecs directly, there would be no need to create a dynamic codec plugin infrastructure in OpenIGTLink). So, to me it looks like overall it would make more sense to not bundle OpenIGTLinkIF with the Slicer core by default.
I talked to @ungi and he would have no objections to move out OpenIGTLinkIF either (he said he always installed SlicerIGT before doing anything anyway, and we could make SlicerIGT extension to depend on OpenIGTLink extension).
As for the IGT category, I remember that we created and kept OpenIGTLinkIF there as a default module because we wanted to make the IGT activities visible in the Slicer community. But that was when we had a lot more default modules in 3D Slicer. Now the users are expected to install extensions almost any type of research and I think the visibility in the default menu became less important.
I agree, OpenIGTLink as extension name sounds good (maybe SlicerOpenIGTLink? but I think OpenIGTLink is enough). Repository name should be SlicerOpenIGTLink to make it clear in GitHub that this is a Slicer extension.
It would build OpenIGTLink and OpenIGTLinkIO libraries using superbuild.
I will make a repo in https://github.com/openigtlink, which i am currently maintaining and have full access to it.
Once the video codec is added into the Slicer core, OpenIGTLink library will be able to link to the codec libraries and video streaming would be available in Slicer.
SlicerVideoCodec could be a solution. The OpenH264 codec provides a link for binary download:
So we should be able to trigger the download in the SlicerVideoCodec extension.
Regarding the other codecs, i am not sure what would be the best method to integrate into slicer.
We should probably add an abstract interface for video compression (and maybe also a trivial uncompressed storage class) into Slicer core and implement interface to register/unregister codecs, similarly to displayable managers.
From a user perspective I think there may be situations where OpenIGTLink is essential and expected despite access to the extension server being limited. Would it make sense to package a Slicer binary with “core extensions” along with a Slicer “Lite” without any?
There are two sides of this issue. If we don’t put OpenIGTLinkIF into an extension, it would be hard to add compressed video support. Not having compressed video is a blocking issue in many projects. On the other hand installing OpenIGTLinkIF from the extension manager or building a custom Slicer packaged with it already is not a impossible thing to do. For the sake of compressed video, I think we should go ahead with the extension plan.
I just create a pull request for SlicerOpenIGTLink extension.
As the codec is not available in the Slicer core yet, in the extension, VP9 codec is super-built for linux/Mac, however, it is not easy to superbuild VP9 in windows platform, so i trigger the download of binary file from this repository for the window platform:
Currently i only uploaded the binary file compiled with “Visual Studio 14 2015”. I think Slicer nightly build is using “Visual Studio 12 2013”, am i right? So in windows build Slicer, video streaming will not be available.
unfortunately, i didn’t find binary distributions from webmproject, would be nice if you guys find binary distrubution for other compilers.
I just pull another request to remove the OpenIGTLink from the core. I test the build on my Mac machine, it seems working, would be great if you have a double check.
The build of OpenIGTLinkIF extension failed unfortunately: http://slicer.cdash.org/viewBuildError.php?buildid=1181930
The reason is the version of python interpreter. To build VP9, i current set the python version to be 2.7, however, the slicer build system only have 2.6.5 installed. Please refer the above link for more information.
Regarding the windows build, the Slicer is building the extension using “Visual Studio 12 2013 Win64”, http://slicer.cdash.org/buildSummary.php?buildid=1181801
So I think once we have the VP9 library available, we should able to make the build successful. I am working with Andras in this issue. Hopefully will fix it soon.
