Hello, I was excited to discover the Curved Planar Reformat module recently. It does 95% of what I need it for. However, the rotation of the images in the straightened volume is sometimes awkward for my purposes. I am reformatting down the centerline of airway in a CT volume, and would like to keep “up” in the reformatted volume to be in as consistent an anatomical direction as possible (for example, anterior). The current module uses Frenet-Serret frames, which makes a lot of sense for generality, but for anatomical interpretability it is more helpful if the output volume avoids twisting because of small wiggles in the curve.
I do see that there is a “Rotation angle” parameter in the existing CPR module, however I believe this applies a constant rotation to the Frenet-Serret frame, which does not solve my problem.
The current code uses vtkSplineDrivenImageSlicer, which calculates the Frenet-Serret frame internally and uses that to set the orientation of the output image slices. What I would prefer would be the ability to supply preferred orientation information somehow.
Theoretically, it seems sufficient to supply one preferred orientation vector, and a second back-up one to use if the the curve is perpendicular to the first orientation vector. Then, the “up” direction in the sliced image would be the projection of the preferred orientation vector into the slice plane. If the slice plane were perpendicular to the preferred orientation vector (resulting in a zero projection), then the back-up orientation vector would be projected into the slice plane instead, and that direction used as “up”. The transverse direction in the sliced images would be given by the cross-product of the curve tangent and the projected “up” vector (just like in the Frenet-Serret frame calculation of the binormal).
In my use case, the preferred orientation would be for “up” to be anterior in the original volume, with a back-up of orientation of “up” being inferior.
I think what I would like should be relatively straightforward to implement, but I have essentially no C++ skill, so I am feeling stuck as to how to proceed. Is there a way I could move forward using python, maybe with a simpler reslicer than vtkSplineDrivenImageSlicer?
Hi - yes, what you describe sounds very doable (I’ve done the same in the past but the code was not open). It sounds like you understand the math well enough that hopefully you could learn the C++ code or recreate the same in python. Below is some similar code in python if you decide to go that route.
Could you show a few screenshots that illustrate unwanted wiggles?
Switching between primary and secondary “up” direction is not trivial (there could be artifacts due to the sudden twist). Have you tried this on curves that had sections with large bends?
In endoscopy module, rotation is constrainex automatically based on plane fitting (“up” vector is the least-square-fit plane normal). Would that work for you? “up” is not aligned with any anatomical direction exactly but it is more robust (less chance of tangent being nearly equal to “up”) and simpler for the user (no need to choose a direction manually). Maybe we can add options for choosing eirher fitted plane normal or one of the anatomical axes.
Looking at the Endoscopy module, I think I may not understand it correctly. I can see that the Endoscopy module fits a plane to all the supplied points, and then finds the normal to that plane, and then I thought it used that normal to determine the camera View Up vector. However, this does not appear to be the case.
Instead, I think the only thing which is changing the camera View Up is the camera OrthogonalizeViewUp() command ( https://github.com/Slicer/Slicer/blob/master/Modules/Scripted/Endoscopy/Endoscopy.py#L282). I think this effectively takes whatever the current camera ViewUp vector is, projects it into the plane orthogonal to the current view, and renormalizes. In the vtkCamera code, there is also an optional UserViewTransform which I don’t understand (https://gitlab.kitware.com/vtk/vtk/blob/master/Rendering/Core/vtkCamera.cxx#L346-361), but at least in the absence of this, I think the camera view up is not based on the fit plane at all. Instead, I think that the endoscopy camera view management is based on minimally changing it from whatever the view up was in the previous view. This makes for smooth changes from frame to frame, but does nothing to privilege any particular direction.
However, I think I now understand enough about how this works to get a version working on my own. If I am successful, I will try to come back and post my solution here.
Yes, this is correct and as intended. We only use the curve normal in the Transform node update; the 3D view camera’s view up is kept as is (only orthogonalized with the view normal direction).
The transform node is computed so that it can be used for reslicing the images (e.g., using SlicerIGT extension’s Volume Reslice Driver module), position models, etc.