How to configure VR controls interactions in Python?

guillaumarie
Sat, 06 May 2023 00:28:54 +0000
Development

Hi,

I’m developing a 3DSlicer module using the virtual reality module and a Valve Index headset.
I’m trying to configure my own controller interactions in order to:

1. Deactivate all default VR controls

2. Add new customized VR controls (link the joystick movements to a transform so that a model moves accordingly // use the L/R triggers as “Previous” and “Next” buttons)

The closest I’ve been is by getting the vtkRenderWindowInteractor3D using slicer.modules.virtualreality.viewWidget().interactor(). The good point is I’m able to remove event observers from this interactor so that the default VR controls don’t work anymore.
But, out of 51 default event observers that I found in the vtkRenderWindowInteractor3D, it is enough to deactivate only 3 of them so that no command works anymore: PinchEvent, Move3DEvent and Button3DEvent (the 48 others observers can be removed and it doesn’t seem to change anything).
This is an issue for my second goal, because as I want to allocate specific VR controls to individual events, it seems that all controls are accessible through 3 observers. Furthermore, I’d like to get data that come from events (particularly 2D vector for joysticks).

Then I found this code from vtkVirtualRealityViewInteractorStyle.cxx

//----------------------------------------------------------------------------
// Copied from vtkOpenVRInteractorStyle::SetupActions
//------------------------------------------------------------------------------
void vtkVirtualRealityViewInteractorStyle::SetupActions(vtkRenderWindowInteractor* iren)
{
  vtkOpenVRRenderWindowInteractor* oiren = vtkOpenVRRenderWindowInteractor::SafeDownCast(iren);

  if (oiren)
  {
    oiren->AddAction("/actions/vtk/in/Elevation", vtkCommand::Elevation3DEvent, true);
    oiren->AddAction("/actions/vtk/in/Movement", vtkCommand::ViewerMovement3DEvent, true);
    oiren->AddAction("/actions/vtk/in/NextCameraPose", vtkCommand::NextPose3DEvent, false);
    oiren->AddAction("/actions/vtk/in/PositionProp", vtkCommand::PositionProp3DEvent, false);
    oiren->AddAction("/actions/vtk/in/ShowMenu", vtkCommand::Menu3DEvent, false);
    oiren->AddAction("/actions/vtk/in/StartElevation", vtkCommand::Elevation3DEvent, false);
    oiren->AddAction("/actions/vtk/in/StartMovement", vtkCommand::ViewerMovement3DEvent, false);
    oiren->AddAction("/actions/vtk/in/TriggerAction", vtkCommand::Select3DEvent, false);
  }
}

But the interactor I use is from the vtkRenderWindowInteractor3D class, which doesn’t have these lines. So I got access to the interactorStyle vtkVirtualRealityViewInteractorStyle using interactor.GetInteractorStyle() but this “object has no attribute ‘SetupActions’”.
Plus, the above vtkCommand are not understood neither by the interactor nor by the interactorStyle.

How should I do? If I’m not on the right track at all, what is the right way to change controls? Can I do this in Python? If not, what are the alternatives?

Sorry if it’s not clear but it’s also not in my mind, I hope I provided enough informations.
Thanks you very much,

[Slicer 4.11.20210226 - Python 3.7]
PS: I’m using an old 3DSlicer version because VR controllers didn’t work in the latest version last time I checked (2023-03-03).

6 posts - 5 participants