Operating system: Windows 10
Slicer version: 4.11.2
Hi,
I am working on a project that involves the incorporation of collision detection capabilities, to be applied to neurosurgical robotics applications.
Currently, the project is at its infancy, where I am attempting to model objects of interest in 3D Slicer and to explore collision detection features and capabilities.
I have built a module in Slicer to serve as a basic framework for such exploratory work. It involves the creation and loading of primitive vtkMRMLModelNode 3D objects into the scene, meant to simulate a neurosurgical setting (sphere = critical brain anatomy, rod = robotic arm).
The user can manipulate either object in 3D space by reorienting it and modifying its position and size. The module interface is as shown:
The goal is to be able to detect the collision as displayed above in the Slicer scene.
The question becomes then: are there any existing techniques, tools, libraries/classes, or extensions that could provide collision detection coverage?
In the SlicerRT extension there is a module that uses collision detection. The filter doing it, and which you can use for your project is vtkCollisionDetectionFilter. There was ongoing work to integrate it in VTK, but I’m not sure if it has been finished. The module using it, and which can serve as an example, is RoomsEyeView.
Thank you very much for the reply and for the examples provided. Work on this project had grinded to a halt due to other competing priorities, but has emerged back to the forefront.
Hence, I would like to pose some follow-up questions:
This is somewhat of a sanity check, but my understanding is that the vtkCollisionDetectionFilter is not a standalone function, i.e., given that it does not seem to be integrated into VTK, I cannot simply call it as I would for say, model creation (e.g. vtk.vtkSphereSource).
Rather, it seems to me that I will have to clone the repo referenced (SlicerRT/SlicerRtCommon), containing all the relevant scripts and files of the vtkCollisionDetectionFilter, and then source them into my project (as was done by the RoomsEyeView module). Is my understanding correct?
The scripts and functions comprising the vtkCollisionDetectionFilter are written in C++, as was the RoomsEyeView example module sourcing it, hence simplifying integration and function calls to collision detection features.
However, my code framework as described in my original post (i.e. consisting of widget instantiation of drop-down menus, sliders, event handling, and population of the Slicer scene with geometrical objects) is written in Python.
As such, this poses integration challenges as I will need to make calls to the C++ vtkCollisionDetectionFilter functions from my Python framework. I have looked into invoking Python bindings such as ctypes and SWIG, but such translation schemes seem somewhat intimidating to a novice like me.
Hence, I am wondering whether there may be a simpler solution that will allow me to leverage the vtkCollisionDetectionFilter functions (written in C++) into my Slicer programming development work that has been done in Python.
Yes it is part of SlicerRT currently. There have been initiatives for integrating it to VTK (see for example here), but I don’t think it has been finished. You can either use SlicerRT, or you can decide to just copy this filter to your project. It would be cleaner to rely on SlicerRT (maintenance in only one place), but if you do not use anything else from there, then it is understandable if you copy these two files to your own purpose.
Thank you very much for the reply and for the guidance provided so far.
Indeed, the line of code import vtkSlicerRtCommonPython works in the Python Interactor (clean return), provided that I have the SlicerRT extension installed:
However, when I try to include this import line in my script, Slicer is not able to recognize it:
I tried modifying the import statement in my script to point it to the SlicerRT repo folder cloned in my workspace as follows: from SlicerRT.SlicerRtCommon import vtkSlicerRtCommonPython. Yet this gives me the following error:
How would I be able to make this import in my script?
As @lassoan pointed out, the filter is available in VTK, simply using vtk.vtkCollisionDetectionFilter().
To answer your question specifically, I suspect that your module is imported before SlicerRT. Try just importing it in the function in which you use it. Also make sure that both SlicerRT and your module are added at the same time (sorry if too trivial but one can miss trivial stuff easily, I know that from experience)
Thank you for your continual help and guidance. Yes, I did look into @lassoan’s comment regarding the filter being available in VTK. Yet I am not sure if I understood it correctly, because it does not work when I tried running it in the Python Interactor:
However, the comment was prefaced by this expecting to work in the latest Slicer Preview release (4.13), yet I am on the stable release version (4.11). So perhaps I will have to upgrade and install the latest Slicer Preview release version and test it out there.
As for the import not being picked up by my module, indeed, it most likely has to do with me not being diligent enough with trivial module management. I am not sure if I understand the comment though, regarding adding SlicerRT and my module (Neurosurgical Robotics) at the same time, but this is my current setup:
Module Script:
My imports are listed out at the top of my script as shown below:
Module Loading Management:
I believe I do have it set up such that my module (CollisionDetection) is being imported after SlicerRT. If I understand correctly, this can be verified by the order in which the two modules are listed in the Application Settings → Modules window:
In addition, my module appears to be loaded in after the SlicerRT extension:
Module Directory Workspace
I have my directory workspace structured as follows: my module script CollisionDetectionFramework.py is contained in the folder collisionDetection, with the SlicerRT project cloned and located in the same folder, such that import accesses to the SlicerRT module can be made:
Hopefully this is enough information for you to be able to diagnose the issue here. Thank you.
Thank you so much for the help and guidance so far. I am now able to access the VTK collision detection filter through my module script as well as make function calls to it!
Currently; however, I am trying to figure out the set of function calls I will need to make through the filter to detect collisions between my simplistic VTK model objects.
I have studied the code in the RoomsEyeView module to get a sense of the function calls needed to set up collision detection. My understanding is summarized as follows:
Instantiate the VTK collision detection filter
Define the models to be passed in as inputs
Set the transforms required
Return the number of contacts detected
Step 3 is where my understanding of the code is weak, given that the SlicerRT environment is much more complicated than mine, with many different models and reference systems. My module is populated with only a sphere and a rod for simulation purposes, and I am not quite sure what transforms I would need to specify or create.
The code block below is what I have tried so far (ignoring the need for setting transforms), and the function call to GetNumberOfContacts() causes Slicer 4.13 to crash:
collisionDetection = vtk.vtkCollisionDetectionFilter() input1 = slicer.mrmlScene.GetFirstNodeByName("Input Model 1") input2 = slicer.mrmlScene.GetFirstNodeByName("Input Model 2") collisionDetection.SetInputData(0, input1.GetPolyData()) collisionDetection.SetInputData(1, input2.GetPolyData()) numContacts = collisionDetection.GetNumberOfContacts()
My assumption is that this code crashes on me because I have not set the transforms required through collisionDetection.SetTransform(), since I do not understand what parameters I would need to pass in or what this function call is doing.
The SlicerRT example seems to set the transforms required by making calls to SlicerIECTransformLogic and calling GetTransformNodeBetween() between the object model of interest and the reference frame RAS. My understanding here is weak though, since I can’t see why this is necessary or what transforms would be needed.
Hence, my follow-up questions are the following:
What transforms would I need to set/pass in to the collision filter (and presumably create beforehand)?
The SlicerIECTransformLogic routines are a bit intimidating since I don’t fully understand what’s going on. How could I go about creating these transforms or getting the necessary transforms between two frames in a simpler manner?
There are many examples on VTK site. The collision example has two spheres: sphere0 is moving and it has a transform0 as a transformation data, sphere1 is stationary and it has simple identity matrix.
Thank you very much for the reply and for the example provided. My case is similar, in that I have a sphere and a cube in the Slicer scene, but they are stationary VTK objects. They can; however, be translated by the user and moved around the scene.
I have tried following the example in how the transforms were set, but I still have Slicer 4.13 crashing on me upon the GetNumberOfContacts() function call.
This is what my code looks like so far:
The transforms transform0 and transform1 are initialized using vtkTransform() and are initially set to the centers of the VTK objects using vtkTransform.Translate(). As the user moves the objects around, I update the transforms for each object using vtkTransform.Translate() to have the transforms fixed to the center of the objects.
I tried printing the output of the collision filter using the GetOutput() call for one of my VTK objects, and this is the debug trace I get:
vtkPolyData (000001E17CFD33F0)
Debug: Off
Modified Time: 418414
Reference Count: 2
Registered Events: (none)
Information: 000001E1569BECA0
Data Released: False
Global Release Data: Off
UpdateTime: 0
Field Data:
Debug: Off
Modified Time: 418411
Reference Count: 1
Registered Events: (none)
Number Of Arrays: 0
Number Of Components: 0
Number Of Tuples: 0
Number Of Points: 0
Number Of Cells: 0
Cell Data:
Debug: Off
Modified Time: 418414
Reference Count: 1
Registered Events:
Registered Observers:
vtkObserver (000001E15699F7D0)
Event: 33
EventName: ModifiedEvent
Command: 000001E1569BEAC0
Priority: 0
Tag: 1
Number Of Arrays: 0
Number Of Components: 0
Number Of Tuples: 0
Copy Tuple Flags: ( 1 1 1 1 1 0 1 1 1 1 1 )
Interpolate Flags: ( 1 1 1 1 1 0 0 1 1 1 1 )
Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
Scalars: (none)
Vectors: (none)
Normals: (none)
TCoords: (none)
Tensors: (none)
GlobalIds: (none)
PedigreeIds: (none)
EdgeFlag: (none)
Tangents: (none)
RationalWeights: (none)
HigherOrderDegrees: (none)
Point Data:
Debug: Off
Modified Time: 418413
Reference Count: 1
Registered Events:
Registered Observers:
vtkObserver (000001E15699F710)
Event: 33
EventName: ModifiedEvent
Command: 000001E1569BEAC0
Priority: 0
Tag: 1
Number Of Arrays: 0
Number Of Components: 0
Number Of Tuples: 0
Copy Tuple Flags: ( 1 1 1 1 1 0 1 1 1 1 1 )
Interpolate Flags: ( 1 1 1 1 1 0 0 1 1 1 1 )
Pass Through Flags: ( 1 1 1 1 1 1 1 1 1 1 1 )
Scalars: (none)
Vectors: (none)
Normals: (none)
TCoords: (none)
Tensors: (none)
GlobalIds: (none)
PedigreeIds: (none)
EdgeFlag: (none)
Tangents: (none)
RationalWeights: (none)
HigherOrderDegrees: (none)
Bounds:
Xmin,Xmax: (1, -1)
Ymin,Ymax: (1, -1)
Zmin,Zmax: (1, -1)
Compute Time: 0
Editable: false
Number Of Points: 0
Point Coordinates: 0000000000000000
PointLocator: 0000000000000000
CellLocator: 0000000000000000
Number Of Vertices: 0
Number Of Lines: 0
Number Of Polygons: 0
Number Of Triangle Strips: 0
Number Of Pieces: 1
Piece: -1
Ghost Level: 0
Interestingly, it seems that all the values are ‘‘null’’, i.e. no geometrical properties of the VTK object seem to be captured or processed, with most values found to be 0.
I would appreciate any debugging help or insight into this matter. Thank you.
Yes, I do. The transforms are updated as the objects are moved around and upon the click of the ‘‘Detect Collision’’ button below, I then call onCollisionDetection() shown above. This then updates the collision filter before calling the GetNumberOfContacts method.
Thank you for the suggestion, I was able to reproduce the VTK example step-by-step in my Slicer environment and it indeed worked.
My scenario is a little different than the VTK example; however, in the sense that the user can move the objects around in the Slicer scene. In contrast, the VTK example has the objects a fixed distance apart and then proceeds to move one of the objects in a predetermined manner.
The MLC position example code was very helpful in revealing a step that I was missing, which was updating the collision filter through the Update() method.
With this fix, Slicer no longer crashes on me when I call GetNumberOfContacts(), but it is not reporting a collision between the objects, i.e. GetNumberOfContacts() returns 0 even though I have the objects oriented in a collision.
I suspect it has to do with the way I am setting my transforms, i.e. I am simply translating each transform to the center of its respective object using Translate(), and that may be inadequate for the purposes of passing it through the collision filter.
This is what my code look like so far, I would greatly appreciate some debug help:
I think you right. You can try to make a gap between your objects, and then use loop to change transform position of one object until it collides with another one. On each iteration of the loop you must update the transform and the collision filter and check number of contacts.
If you call GetPolyData() or GetMesh() on a model node then it returns the mesh in the node’s local coordinate system, not transformed into the world coordinate system.
If you only apply linear transforms to your model nodes then you can retrieve that transform from the parent transform node of each model node and set that in the collision detection filter. This way collision detection is performed on the exact same arrangement that you see in the viewers.
Yes, this is the crux of the problem, in that I do not know how to extract the transforms from the model nodes so that I can pass them into the collision detection filter.
My application has model nodes being translated across the screen by the user, with the X, Y, and Z center coordinates updated accordingly using SetCenter(). How then would I be able to extract the resultant translations or the respective transforms associated with each model node?
The examples that I were provided with on the VTK site and which I tried to follow consisted of initializing transforms, and then updating these transforms according to the translations, and then passing them through the filter. Yet there was a disconnect to me in how these transforms were related to or enforceable on the model nodes.
Therefore, what I need help in is figuring out what transforms to pass into the collision filter and how I can go about extracting them. Thank you!
