# Stereo Off-Axis Projection and Mouse Interactions

**URL:** <https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552>\
**Category:** Support\
**Tags:** python, scripted, extensions\
**Created:** [July 6, 2026, 6:31pm UTC](https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552 "2026-07-06T18:31:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![LucasP](https://avatars.discourse-cdn.com/v4/letter/l/b9bd4f/32.png) [@LucasP](https://discourse.slicer.org/u/LucasP)\
**Post date:** [July 6, 2026, 6:31pm UTC](https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552/1 "2026-07-06T18:31:32Z")

</div>

> 🤖 This post was written with AI assistance. But I’m still human

Hi,

I’m new to 3D Slicer and VTK. I am trying to build a “virtual window” effect using a custom stereo display that gives me real-time `viewDistance` and `eyeSeparation`.

My goal is to let the user interact normally with the 3D volume using standard mouse controls (rotate, pan, zoom). From my understanding, to achieve this “virtual window” effect while preserving standard mouse interactions, the physical screen coordinates must dynamically track the camera: the screen center should sit exactly at the camera position, and the `EyePosition` should be placed behind it (`camera.GetPosition() - viewDist * dop`).

However, as soon as `camera.SetUseOffAxisProjection(True)` is invoked, it feels like `ViewUp` is locked, and the user can no longer rotate the camera, only translate it. This leads to broken math on my side during mouse movements.

Here is how I’m updating the frustrum :

```auto
  layoutManager = slicer.app.layoutManager()
  threeDWidget = layoutManager.threeDWidget(0)
  renderWindow = threeDWidget.threeDView().renderWindow()
  renderer = renderWindow.GetRenderers().GetFirstRenderer()
  camera = renderer.GetActiveCamera()
  p = self.stereoParams
  pixelPitch = p['pixelPitch'] # unit : meter
  viewDist = p['viewDistance'] # Distance User to screen (m)
  eyeSeparation = p['eyeSeparation'] # IPD (m)

  viewportSizePx = renderWindow.GetSize()
  if viewportSizePx[0] <= 0 or viewportSizePx[1] <= 0:
      return # Window minimized or invalid
  
  viewportWidth = viewportSizePx[0] * pixelPitch # viewport width (m)
  viewportHeight = viewportSizePx[1] * pixelPitch # viewport height (m)
  halfW = viewportWidth / 2.0
  halfH = viewportHeight / 2.0

  up = np.array(camera.GetViewUp())
  dop = np.array(camera.GetDirectionOfProjection())
  cam = np.array(camera.GetPosition())

  right = np.cross(dop, up)
  rMag = np.linalg.norm(right)
  right /= rMag

  bl = cam - halfW * right - halfH * up
  br = cam + halfW * right - halfH * up
  tr = cam + halfW * right + halfH * up

  camera.SetUseOffAxisProjection(True)
  camera.SetScreenBottomLeft(bl)
  camera.SetScreenBottomRight(br)
  camera.SetScreenTopRight(tr)
  camera.SetEyePosition(camera.GetPosition()- viewDist * dop)
  camera.SetEyeSeparation(eyeSeparation)

```

It seems that I’m not doing things right.  
Is there any documentation or existing examples on implementing off-axis projection with dynamic screen coordinates within Slicer?

Thanks for any help!

---

<div class="post-metadata">

**Author:** ![mau\_igna\_06](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/mau_igna_06/32/9056_2.png) [@mau\_igna\_06](https://discourse.slicer.org/u/mau_igna_06)\
**Post date:** [July 7, 2026, 2:21pm UTC](https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552/2 "2026-07-07T14:21:56Z")

</div>

From a quick read to the code I think you forgot to use the set methods for the camera (e.g. `SetViewUp`, `SetDirectionOfProjection`, `SetPosition`)

---

<div class="post-metadata">

**Author:** ![LucasP](https://avatars.discourse-cdn.com/v4/letter/l/b9bd4f/32.png) [@LucasP](https://discourse.slicer.org/u/LucasP)\
**Post date:** [July 7, 2026, 2:57pm UTC](https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552/3 "2026-07-07T14:57:39Z")

</div>

Hi Mauro,

Thanks for the reply!

From my undestanding Slicer’s mouse interactor is dynamically updating the camera’s `Position`, `ViewUp`, and `DirectionOfProjection` as the user interacts, and my code reads those updates to dynamically recalculate the screen coordinates.

The core issue is that as soon as `SetUseOffAxisProjection(True)` is enabled, the mouse interactor loses the ability to rotate the camera and the `ViewUp` vector becomes completely locked. Only translation continues to work.

It feels like VTK’s internal off-axis calculations are fighting or overriding the interactor’s ability to modify the camera orientation. Is there a specific way to configure the interactor style or the camera so that rotation updates are still processed properly in off-axis mode?

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [July 13, 2026, 11:18am UTC](https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552/4 "2026-07-13T11:18:19Z")

</div>

If you just want to do since quick testing in python then you can add an observer with high priority that captures and consumes mouse move and button events. If you want to have a proper solution then you probably need to investigate what happens exactly in the mouse event handler in C++.

---

<div class="post-metadata">

**Author:** ![Tomas\_Gomes](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/tomas_gomes/32/82618_2.png) [@Tomas\_Gomes](https://discourse.slicer.org/u/Tomas_Gomes)\
**Post date:** [July 14, 2026, 6:22pm UTC](https://discourse.slicer.org/t/stereo-off-axis-projection-and-mouse-interactions/47552/5 "2026-07-14T18:22:48Z")

</div>

Hello Mr.Andras Lasso!

My name is Tomás. I´m a Medical Student from Portugal and I´m currently exploring the 3D Slicer app and it´s extensions for CT scan images of the spine. I am new on this 3D Slicer forum, so I wasn´t able to send you a message. Is it possible to ask you some questions via e-mail or even in this forum through direct messaging if feasible? Thank you for your time in advance!