Transition of nightly build from VTK 9.0.20201111 to 9.1.20220125

The pull request updating VTK is ready for review (see PR-5978)

It fixes issue #5956: Update VTK9 to the latest master

2022.01.27 Experimental build for PR-5978 have been started. See dashboard :white_check_mark:
2022.01.28 Upload packages in Google drive folder 3D Slicer Experimental Packages :white_check_mark:

Acknowledgments

This was made possible by the contributions and help of many :pray:

More specifically, I would like to acknowledge the specific help of @Sankhesh_Jhaveri , dgobbi, ben.boeckel, @lassoan, @Connor-Bowley, will.schroeder who helped track down & address issues, as well as @jadh4v who helped test & finalized the integration.

Status of the transition

Slicer Application

After the pull requests list below are integrated, Slicer application will be expected to be successfully build and package on Linux, macOS and Windows.

description merged?
PR-6059 COMP: Update SystemTools use anticipating update from VTK 9.0 to 9.1 :white_check_mark:
PR-5978 ENH: Update VTK9 from 9.0.20201111 to 9.1.20220125 :hourglass_flowing_sand:

Help needed to confirm there are no regressions

Check the following:

  • Markups module
  • Volume rendering
  • Coincident topology

Help create a check list ? Including reference to previous issue reports.

Issues to reevaluate once update is complete:

Slicer Extension support

Remaining tasks will be the following:

1 Like

Experimental packages are available for download here: 3D Slicer Experimental Packages

Additional failing tests building against VTK 9.1.20220125

computron (macOS)
vtkMRMLThreeDReformatDisplayableManagerTest1 (segfault)
metroplex (Linux)
vtkMRMLThreeDReformatDisplayableManagerTest1 (segfault)

Output of vtkMRMLThreeDReformatDisplayableManagerTest1

overload (windows)

Source: https://slicer.cdash.org/test/21156757

The regression has been fixed in Slicer/VTK in e0d1851e2 and a merge request in being review in upstream VTK at BUG: Rename vtkInteractionCallback class to avoid symbol clash (!8885) · Merge requests · VTK / VTK · GitLab

Impact of the regression on Slicer application

Enabling the “Lock Normal to Camera” associated with the reformat widget was leading to a crash.

image

Details

After hunting down the regression (performing a manual bisection of the history), I discovered that the following commits introduced the regression:

kitware/vtk@7451dd2d1: vtkDisplaySizedImplicitPlaneWidget/Representation: Implementation

kitware/vtk@cd42f785d: vtkCoordinateFrameWidget: Implementation

At first, it didn’t make sense because we do not instantiate these new widgets in Slicer.

Looking at the issue more closely, I realized that the commit referenced above introduced the class vtkDisplaySizedImplicitPlaneWidget based of vtkImplicitPlaneWidget2 that we effectively used in Slicer (see here).

It turns out that these classes all defined the class vtkInteractionCallback in their private implementation along with:

vtkImplicitPlaneWidget2 vtkDisplaySizedImplicitPlaneWidget vtkCoordinateFrameWidget
static constructor link link link
friend statement in header link link link

Since these classes were compiled within the same “program” (in that case the shared library libvtkInteraction-9.1.(dll|dylib|so)), the ODR (One Definition Rule) was violated and it now explains the following backtrace:

Backtrace obtained runing ctest -R vtkMRMLThreeDReformatDisplayableManagerTest1 -V and analyzing the core dump following the Slicer guide C++ debugging on GNU/Linux systems: Analyze a segmentation fault.

Symbol analysis

The following invocation of nm allows to confirm there the patch effectively ensure there are different instances of callback classes instead of a single vtkInteractionCallback:

Before integrating our fix:

$ nm lib/libvtkInteraction-9.1.so  | c++filt | ack "typeinfo for vtk.+Callback$"
0000000000501f70 d typeinfo for vtkCWCallback
000000000050b938 d typeinfo for vtkPWCallback
000000000050b950 d typeinfo for vtkPW1Callback
000000000050b968 d typeinfo for vtkPW2Callback
00000000004fae80 d typeinfo for vtkAngleWidgetCallback
00000000004f7a28 d typeinfo for vtkImageViewerCallback
0000000000503d88 d typeinfo for vtkInteractionCallback
00000000004f7c10 d typeinfo for vtkImageViewer2Callback
0000000000500f70 d typeinfo for vtkCaptionAnchorCallback
0000000000505d58 d typeinfo for vtkDistanceWidgetCallback
00000000004fca38 d typeinfo for vtkBiDimensionalWidgetCallback
00000000004f7f28 d typeinfo for vtkResliceImageViewerScrollCallback

After integrating our fix:

$ nm lib/libvtkInteraction-9.1.so  | c++filt | ack "typeinfo for vtk.+Callback$"
0000000000501df0 d typeinfo for vtkCWCallback
000000000050b948 d typeinfo for vtkPWCallback
000000000050b960 d typeinfo for vtkPW1Callback
000000000050b978 d typeinfo for vtkPW2Callback
00000000004fad00 d typeinfo for vtkAngleWidgetCallback
00000000004f78a8 d typeinfo for vtkImageViewerCallback
00000000004f7a90 d typeinfo for vtkImageViewer2Callback
0000000000500df0 d typeinfo for vtkCaptionAnchorCallback
0000000000505ca0 d typeinfo for vtkDistanceWidgetCallback
00000000004fc8b8 d typeinfo for vtkBiDimensionalWidgetCallback
00000000004f7da8 d typeinfo for vtkResliceImageViewerScrollCallback
000000000050aa80 d typeinfo for vtkImplicitPlaneWidget2InteractionCallback
0000000000503c08 d typeinfo for vtkCoordinateFrameWidgetInteractionCallback
0000000000504518 d typeinfo for vtkDisplaySizedImplicitPlaneInteractionCallback

Running test interactively disabling event replay

For future reference, it has also been useful to run the test interactively by disabling the replay of event, this was done by appending -I --DisableReplay to the command reported running the test with -V:

SLICER_SOURCE_DIR=/home/jcfr/Projects/Slicer
SLICER_BUILD_DIR=/home/jcfr/Projects/Slicer-rwdi

${SLICER_BUILD_DIR}/Slicer-build/Slicer --launch \
  ${SLICER_BUILD_DIR}/Slicer-build/bin/MRMLDisplayableManagerCxxTests \
  vtkMRMLThreeDReformatDisplayableManagerTest1\
    -D ${SLICER_SOURCE_DIR}/Libs/MRML/DisplayableManager/Testing/Cxx/../ \
    -T ${SLICER_BUILD_DIR}/Slicer-build/Testing/Temporary \
    -V Baseline/vtkMRMLThreeDReformatDisplayableManagerTest1.png \
    -I --DisableReplay

Further analysis of the VTK toolkit

Running the following commands allowed to discover other duplication and these ones will be discussed with the team as well:

  • #18456 (Remove duplicated class vtkTextureArray)
  • #18455 (Remove duplicated class vtkmOutputFilterPolicy)
  • #18457 (Improve continuous integration to detect duplicated symbols)
$ cd VTK
$ ack -h --ignore-dir=Testing "^class.+vtk.+ \: " | ack "\:" | sort  > /tmp/vtk-classes.txt
$ ack -h --ignore-dir=Testing "^class.+vtk.+ \: " | ack "\:" | sort | uniq > /tmp/vtk-uniq-classes.txt
$ diff /tmp/vtk-classes.txt /tmp/vtk-uniq-classes.txt
diff /tmp/vtk-classes.txt /tmp/vtk-uniq-classes.txt
1481,1482d1480
< class vtkInteractionCallback : public vtkCommand
< class vtkInteractionCallback : public vtkCommand
2047d2044
< class vtkmOutputFilterPolicy : public vtkm::filter::PolicyBase<vtkmOutputFilterPolicy>
2607d2603
< class vtkTextureArray : public std::map<int, vtkSmartPointer<vtkImageData>>

Occurrences of vtkmOutputFilterPolicy:

Occurrences of vtkTextureArray:

4 Likes

Impressive work Jc! Glad you were able to track this down :pray: