Despite the expectations about all the fixes done in VTK since the version Slicer used to be stuck on, unfortunately it seems that vtkQWidgetWidget cannot be used with Slicer built against VTK9.
First I tried the code that worked “more or less” (i.e. the Qt widget appeared in the VTK renderer, although there were rendering issues and it crashed when the widget was created on the heap instead of the stack) - it can be found in this SlicerVR branch.
Then I wanted to get rid of the complications to see what was going on and created this minimal Slicer extension that has one module exposing one button which is reproducing the VTK test.
It crashes when enabling the QWidgetWidget, in the nvoglv64.dll module. In the main thread the call stack varies, but it is sometimes at the destruction of the widget (i.e. it reaches the end of the addHelloWorldClicked function), but many times in vtkOpenGLFramebufferObject::Blit, within process_events_and_wait.
This widget seems to work in VTK and in Paraview. Am I doing something wrong with how I use it? Any suggestions about what to try to get it working? Thank you very much!
This call process_events_and_wait looks like a red flag to me, since processing events could trigger a render or other changes to the qt or opengl state. Is that being used in ParaView? Maybe it’s okay in the vtk testing code since there are no other events going on, but usual Qt style would be have everything be event driven.
I agree that we should not pump the event loop by calling process_events_and_wait, but it should be done by the application as usual.
I would add that as far as I remember, ParaView’s implementation did not allow simultaneous virtual reality+desktop experience (at least this was the first implementation that I tried several years ago), but if you start virtual reality mode then the desktop application GUI is blocked. So, Slicer might encounter some issues that ParaView does not.
I had a look at the vtkQWidgetWidget implementation and it uses the same messy state and event design as other VTK core widgets, so we will need to have our own modified clone of the widget and representation in Slicer. As a first step, you could test just porting the representation and place it in the renderer (it is just an actor) and see if it works. If it does then we know that the crash is most likely due to the message handling.
Process events: Of course I tried first without this (see the SlicerVR branch linked in the post above), and the application crashes if I just add the widget without any force update. I only used the VTK testing code to demonstrate that the same thing that they use in VTK to test the widget with crases in Slicer environment.
@lassoan By porting the representation do you mean migrating the widget from VTK to the widget-representation mechanism used lately in Slicer?
Yes, at least change the base class to vtkMRMLAbstractWidgetRepresentation and implement basic functions, such as update from MRML (e.g., control show/hide by observing the corresponding node).
I started porting to the Slicer VTK widget infrastructure. I made some commits in the minimum working example extension with my intermediate results. The second button in the module UI (in Examples category) adds the widget to the 3D view as a quick test.
The example does not work currently with the texture applied. After commenting out this line the widget shows up as a white plane, but uncommenting the line causes the widget to not appear, or show fully transparent, not sure. @lassoan do you have any ide why the texture does not work? I can continue working on this early next week. Thank you!