Customizing 3DSlicer for Video Stream Rendering: Combining ScriptModule for GUI and LoadableModule for Stream Display Without GUI

Development
1

Regarding custom development of 3DSlicer, I am still a newbie and constantly exploring and learning. From the official documentation of 3DSlicer, we can see that there are three types of Modules in 3DSlicer:

  • LoadableModule
  • ScriptModule
  • CLIModule

Currently, we want to read the video stream from a capture card and continuously render it in the SliceView of 3DSlicer in our customized version. This is similar to the SlicerOpenIGTLink module, but without the involvement of a GUI, where all node creation is done automatically.

Since the application itself also involves GUI development, we want to add two Modules to accomplish this:

  • A ScriptModule to handle GUI development.
  • A LoadableModule to handle video stream display.

After reviewing some LoadableModule codes, I found that they are all tied to the GUI. Therefore, I would like to ask:

  1. Is it possible to develop a LoadableModule without a GUI?
  2. Are there any similar cases that we can refer to?
2

The two requirements “render it in the SliceView” and “develop a LoadableModule without a GUI” seems to be contradictory.

By develop a LoadableModule without a GUI, do you mean a module in which you would customize the layout and hide unwanted element like default Slicer toolbars and alike ?

3

Thank you very much for your response.

Although it may seem contradictory, this module will be developed in C++ to ensure processing efficiency, so it does not fall under the scope of ScriptModule. The current design idea is that reading the video stream is an automatic behavior, which means that the video stream is automatically read after the screen transition or when the application starts. Then, the screen refresh is triggered by the update of the node. This behavior indeed looks more like a Non-GUI Loadable Module.

Due to my limited experience with 3DSlicer development, I am currently conducting a feasibility study to ensure that the design plan can be consistent with the existing logic of 3DSlicer.

By develop a LoadableModule without a GUI , do you mean a module in which you would customize the layout and hide unwanted element like default Slicer toolbars and alike ?

Could you provide more specific information on the point you mentioned? Sorry, I have just started looking into 3DSlicer’s code, and it is indeed very large and complex.

4

@herryliq you could start by looking into this feature that already exists. It could simplify your work a lot:

5

@pieper We have also evaluated an implementation plan based on SlicerOpenIGTLink. Considering our requirement for low-latency display of captured video streams, our goal is to minimize the performance overhead in the transmission of video streams before rendering.

Therefore, we are currently inclined to directly embed a video stream reading module in 3DSlicer.By emulating the mechanism of SlicerOpenIGTLink, each captured video frame will be rendered into the window.

6

You can definitely write a custom module in C++ as a loadable module to be as efficient as possible but then do your GUI in Python. I haven’t done it in a while, but you should just be able to set the hidden property to true in the loadable module.

As a general rule though, it’s good to start with the simplest solution and then optimize once you know what the bottlenecks are. If you use OpenIGTLink on the same machine (not over the network) there may not be much overhead.