Highlighted VTK changes associated with Slicer 4.10: Virtual Reality, Rendering, Volume Rendering, Python Wraping, OpenGL performance and threading management

Development
1

This post highlights some of the recent VTK changes included in Slicer 4.10

Blogs

Build

  • BUG: Common naming of #define conflicts with ITK
    kitware/vtk@10873ec (from Hans Johnson <hans-johnson[at]uiowa.edu>)

    When VTK and ITK are used together, the
    #define ThreadInfoStruct vtkMultiThreader::ThreadInfo
    from VTK/Common/Core/vtkMultiThreader.h

    would clobber the ThreadInfoStruct definition from
    ITK/Modules/Core/Common/include/itkMultiThreaderBase.h

    See https://discourse.itk.org/t/vtk9-and-itk5-conflicts/802

    Slicer: Ensure that future build of Slicer against ITK 5.0 and VTK succeed.

Chart

  • Add single click selection in chartXY
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4531/commits (from Davide Punzo <punzodavide[at]hotmail.it>)

  • Fixed control points clamp function to handle NaN value
    kitware/VTK@bc627d8 (from Csaba Pinter <csaba.pinter[at]queensu.ca>)

    If the user clicks in a plot that has an invalid (0,0) range, then in vtkContextScene::ProcessItem that
    is called after mouse events, the mapped mouse position becomes NaN, due to invalid matrix in the ContextScene’s
    transform. This NaN position is then added to the function as a control point with an invalid NaN value. This fix
    makes sure this does not happen, by clamping the NaN values to minimum bounds on x axis, and 0 on y axis (any
    value comparison returns false if an operand is NaN, so need to check explicitly).

    Slicer: Fix issue 4518: Slicer crashes when clicking on Transforms color widget

IO

  • ENH: Improve scene exporters and STL, OBJ, and PLY file IO
    kitware/VTK@3b814e9 (from Andras Lasso <lasso[at]queensu.ca>)

    • Allow reading/writing custom comments in STL, PLY, and OBJ files. This allows
      storing metadata, such as coordinate system, unit, or color in the files.
    • Allow reading/writing binary comments (that can contain 0 character) for STL
      files. This allows reading/writing color information in Mimics-style (Mimics
      software writes RGBA color as 4 bytes in STL file binary header).
    • Allow specifying renderer for exporting: Some exporters rejected to export
      a scene when more renderers were associated with a render window. This was an
      issue because it prevented scene export when additional renderers were used for
      displaying various annotations on the render window. Instead of hardcoding using
      the first renderer, added a ActiveRenderer member, which defines which renderer
      content should be exported. If not set then the first renderer is used, so the
      behavior is backward-compatible.
    • Fixed writing of of .mtl file path in .obj file (full path of .mtl file was
      written into the .obj file).

    Slicer: Used in Segmentation and SegmentEditor modules

  • Add support for writing larger unstructured grids
    kitware/VTK@a7988f5 (from Ken Martin <ken.martin[at]kitware.com>)

    Patch from gitlab Constantine @Butakoff

    Slicer: used in vtkMRMLModelStorageNode and ProbeVolumeWithModel

OpenVR

Performance

  • Resolve "Multithreader creates many unnecessary threads"
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4175/commits (from David Gobbi <David Gobbi <david.gobbi[at]gmail.com>)

    Creating threads is expensive, so creating threads that do nothing should be avoided.

    • Limit number of threads in vtkImageHistogram
    • Remove redundant UpdateExtent calculation.
    • Use vector to allocate arrays instead of new
    • 17279: Limit num threads to num pieces.

  • hardware picking code: improve picking performance and memory footprint
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4175/commits (from Ken Martin <ken.martin[at]kitware.com>)

    Significant rework of the hardware picking code.

    Previously each mapper had to make sure that the rendered
    colors were correct for point/cell etc ID during hardware picking
    and this required large datastructures and texture uploads
    to the GPU on each pick.

    Now it collects the color buffers and gives the mappers a chance to
    update them. This allows us to use gl_VertexId and gl_PrimitiveId
    directly in the shader and then if picked, allow the mapper to
    adjust the color buffer as needed. This allows us to avoid
    rebuilding the VBO and textures each time and avoids the memory
    footprint related to that.

  • Fix issue where picking happened with a bad context
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4616/commits (from Ken Martin <ken.martin[at]kitware.com>)

    Make sure our OpenGL context is current when hardware picking

  • VolumeRendering: update volume rendering to share jitter and depth textures
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4416/commits (from Ken Martin <ken.martin[at]kitware.com>)

  • some opengl performance fixes
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4389/commits (from Ken Martin <ken.martin[at]kitware.com>)

    mostly cache a few items to avoid glGet* invocations

    remove an unused ivar form the image slice mapper

    make it so that the image slice mapper does not force
    an IBO rebuild every frame as typically it is not needed
    and a waste of CPU-GPU

  • Improve vtkAppendFilter efficiency
    kitware/VTK@d36b129 (from Andrew Bauer <andy.bauer[at]kitware.com>)

    If there’s a single unstructured grid in the composite dataset we
    now just do a shallow copy of that to the output.

    Slicer: used in vtkSlicerTransformLogic, vtkSlicerSegmentationsModuleLogic,
    vtkOrientedImageDataResample, vtkMRMLVolumeNode, MergeModels CLI,
    and ScriptedSegmentEditor templates

Python Wrapping

QVTKOpenGLWidget

Rendering:

  • Add support for defining custom uniform variables. See https://gitlab.kitware.com/vtk/vtk/merge_requests/4621
    Thanks: Simon Drouin <drouin.simon[at]gmail.com>

  • Fix issues with order independent translucency_fixes See https://gitlab.kitware.com/vtk/vtk/merge_requests/4757
    Thanks: Ken Martin <ken.martin[at]kitware.com>

  • Add ability to share data between render windows
    kitware/VTK@363d99b

    This topic adds a capability to share some data between
    render windows. Basically it is implemented shared context
    data in OpenGL such as with wglShareLists or the similar
    functionality on glX or Cocoa. Right now only the VBO
    cache makes use of the shared space but in the future shader
    programs, textures, etc could also be shared.

  • vtkOpenGLPolyDataMapper:limit timing calls to reasonable sized renders
    kitware/VTK@f128754

    The timing calls require a round trip to the driver
    which for many actors can be a significant cost. This
    change makes it so that for small datasets the timers
    are called less frequently but still at least once
    per hundred renders.

  • vtkOpenGLRenderWindow: Improve error message reported when OpenGL is not supported.
    kitware/VTK@51e0dc9

    Unable to find a valid OpenGL 3.2 or later implementation.
    Please update your video card driver to the latest version.
    If you are using Mesa please make sure you have version 11.2 or
    later and make sure your driver in Mesa supports OpenGL 3.2 such
    as llvmpipe or openswr. If you are on windows and using Microsoft
    remote desktop note that it only supports OpenGL 3.2 with nvidia
    quadro cards. You can use other remoting software such as nomachine
    to avoid this issue.

    See https://issues.slicer.org/view.php?id=4252

  • Add centralized lighting uniforms and benchmark
    kitware/VTK@84e87cf

    profiling has shown that a lot of time is spent in the
    mapper setting the lighting uniforms when there are a
    large number of mappers. In a more traidiotnal program
    this is handled by a UBO or similar set once per program.
    This topic addds in capability to the shader program
    to store an mtime associated with a group of uniforms.
    It also makes use of that capability by having the renderer
    generate shader code for the lights, and update the unifoms
    when requested and needed. The result is a significant reduction
    in the time spect updating the lights when faced with many mappers.

    This topic also includes a new benchmark to test the performance
    of many actors.

    This topic also includes a performance fix to short circuit
    the mtime check for RenderPasses in the mapper. The most common
    case is none and yet that case was taking significant time.

  • OpenGL2: Check that context exists before trying to pop context.
    kitware/VTK@a7988f5

    On some linux drivers (such as nvidia version 384.111 and 387.34) setting the
    context to zero causes a segfault so check before setting in cases where the
    destruction may be trying to pop to a zero context.

Volume Rendering Fixes

  • Fix volume clip bug and add regression test.
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4338/commits (from Allison Vacanti <allison.vacanti[at]kitware.com>)

    When clipping a volume using in-shader clipping planes, it was possible for the starting point of the ray cast to
    lie beyond the data volume. The raycast code is written such that the first sample is always taken before testing
    termination criteria, and in these cases we would always take a single sample outside of the volume, leading to
    artifacts.

    Fixed this behavior by checking that the starting position calculated by AdjustSampleRangeForClipping is indeed
    inside of the volume bounds and aborting the raycast if it is not.

    The existing TestGPURayCastClipping test would have caught this, except that the vase.vti volume used for testing
    has all 0’s at the boundaries, so the rendering was correct even with the edge-clamp repetition outside of the volume
    (the faulty samples always computed RGBA=vec4(0)). I replaced the vase.vti of this test with a wavelet with finite
    boundary values that will catch this problem if there’s a regression.

  • Fix box widget face highlighting.
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4573/commits (from Allison Vacanti <allison.vacanti[at]kitware.com>)

    The actual data was updated correctly in the BoxWidget
    code, but the wrong object was being marked as modified.

    The rendering code got smarter at some point and now
    only rebuilds VBOs when the corresponding cell array
    has been modified. Marking the entire polydata as modified
    is no longer sufficient to update the on-device
    representation of the highlighted face.

    This patch fixes the BoxWidget so that the active face is
    used for highlighting instead of getting “stuck” on an
    arbitrary face.

  • Volume peel widget flicker
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4568 (from Allison Vacanti <allison.vacanti[at]kitware.com>)

    Correctly apply jittering in ClampToSampleLocation. We were reapplying it inconsistently in the DDP. This
    pass was updated also.

    Rearrange cast initialization code in DDP. The start point was adjusted for clipping after clamping
    to a sample position. This would move the effective sample positions to an inconsistent locations if clipping planes
    were present.

  • Introduce volume API to specify intensity for clipped voxels
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4627 (from Sankhesh Jhaveri <sankhesh.jhaveri[at]kitware.com>)

    See Slicer issue 4513

    Specifying a fixed intensity value for voxels in clipped space provides
    the ability to change lighting computations at the clipped face.

  • VolumeMapper: Invoke update shader event each frame
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4614 (from Sankhesh Jhaveri <sankhesh.jhaveri[at]kitware.com>)

    This allows applications to modify uniforms on the dynamically created and compiled shader program.

  • Fix volume clip bug and add regression test.
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4338/commits (from Allison Vacanti <allison.vacanti[at]kitware.com>)

    When clipping a volume using in-shader clipping planes, it was possible for the starting point of the ray cast to
    lie beyond the data volume. The raycast code is written such that the first sample is always taken before testing
    termination criteria, and in these cases we would always take a single sample outside of the volume, leading to
    artifacts.

    Fixed this behavior by checking that the starting position calculated by AdjustSampleRangeForClipping is indeed
    inside of the volume bounds and aborting the raycast if it is not.

    The existing TestGPURayCastClipping test would have caught this, except that the vase.vti volume used for testing
    has all 0’s at the boundaries, so the rendering was correct even with the edge-clamp repetition outside of the volume
    (the faulty samples always computed RGBA=vec4(0)). I replaced the vase.vti of this test with a wavelet with finite
    boundary values that will catch this problem if there’s a regression.

  • Fix box widget face highlighting.
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4573/commits (from Allison Vacanti <allison.vacanti[at]kitware.com>)

    The actual data was updated correctly in the BoxWidget
    code, but the wrong object was being marked as modified.

    The rendering code got smarter at some point and now
    only rebuilds VBOs when the corresponding cell array
    has been modified. Marking the entire polydata as modified
    is no longer sufficient to update the on-device
    representation of the highlighted face.

    This patch fixes the BoxWidget so that the active face is
    used for highlighting instead of getting “stuck” on an
    arbitrary face.

  • Refactor in-shader volume clipping to be depth-peeling-friendly.
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4071/commits (from Allison Vacanti <allison.vacanti[at]kitware.com>)

    Refactor the volume clipping code so that it is compatible with the volume peeling
    implementation in vtkDualDepthPeelingPass. At a high level, the new version adjusts
    the sampling ray so that it won’t sample clipped areas, rather than check for clipping
    at each sample location.

    This should give better performance (less work during casting) and allows the clip
    calculations to be aware of the depth-peeling sample range.

    kitware/vtk@11c0ccc (Fix early termination criteria in volume peeler)
    kitware/vtk@410f979 (Ensure that the dual depth peeling algorithm samples consistently)
    kitware/vtk@f144205 (Refactor in-shader volume clipping to be depth-peeling-friendly)

    Slicer: Fix issue 4510: VTK OpenGL2 Backend: Cropping is broken with GPU Volume rendering
    if depth peeling is enabled.
    See https://issues.slicer.org/view.php?id=4510

Widgets

  • Invoke DeletePointEvent before deleting vtkSeedWidget seed
    https://gitlab.kitware.com/vtk/vtk/merge_requests/4493/commits (from Sankhesh Jhaveri <sankhesh.jhaveri[at]kitware.com>)

  • Fix picking through a disabled widget with picking manager on
    kitware/VTK@8e4f2f7 (from Johan Andruejol <johan.andruejol[at]kitware.com>)

    With picking manager on, it was impossible to pick a widget if a disabled
    widget was in front of it.
    To fix this, we have:
    - In the old style widgets (derived only from vtkInteractorObserver), we
    went through all the implementation and added the unregistration/registration
    of the pickers in the SetEnabled() method.

    • In new style widgets (derived from vtkAbstractWidget and
      vtkWidgetRepresentation) the method RegisterPickers and UnRegisterPickers
      have been moved into the representation public API to be able to be called
      by the widget’s SetEnabled() method. This allows the widget to register and
      unregisters the pickers as necessary when enabled and disabled.

    In both cases we also implemented the SetPickingManaged method as previously
    the PickingManaged property was never used anywhere in the code base.
    Similarly, the PickersModified() method was removed as its purpose seemed
    redundant to SetPickingManaged().

    The test TestPickingManagerSeedWidget2 was added to demonstrate/test the
    picking behind a disabled widget.

    For more background information, see https://issues.slicer.org/view.php?id=3808

    Co-Authored-by: Ken Martin <ken.martin[at]kitware.com>
    Co-Authored-by: Jean-Christophe Fillion-Robin <jchris.fillionr[at]kitware.com>
    Thanks: Steve Pieper <pieper[at]bwh.harvard.edu>

    Slicer: Allow to remove workaround and Slicer/VTK specific patch associated with Slicer issue #3808

  • BUG: Uniformize RegisterPickers() method
    kitware/vtk@fc043da5b (from Johan Andruejol <johan.andruejol[at]kitware.com>)

    Uniformize the RegisterPickers() method across all the widgets to make sure
    to check for the validity of the picking manager.
    This completes the work started in 8e4f2f7c.

    Slicer: Follow up of issue #3808
    See https://issues.slicer.org/view.php?id=3808

2 Likes