How to map a projector's image to a model(AR) precise?

Support
1

How to map a projector’s image to a 3d printing model of brain(AR) precise?

2

Towards portable image guidance and automatic patient registration using an RGB-D camera and video projector

I read this paper and want to project the points to a model but It is not so easy to. The image is too smal and I need to take the projector move to let the image overlap.

3

You’ll need some way to determine the pose of the 3D printed object relative to the projector - in the paper we start by finding the pose of the object relative to a depth camera. This requires extracting a point cloud of the object from a depth image (module for that here) and performing ICP registration between a model of the object and the object point cloud (we used the SlicerIGT Model Registration module). The position of the depth camera relative to the projector can be found using the ProCamCalib tool. Once the target point on the object in the projector’s coordinate system is known, the 3D target point must be converted to 2D pixel coordinates. You can then generate an image with a small circle at this position and send it to the projector.

Let me know if there’s a specific part of the paper’s workflow I can clarify.

2 Likes
4

Thank you very much. I still confused by this:The 3D target point must be converted to 2D pixel coordinates.

  1. I donnot know how to read (x,y) coordinate in the 3d view(or just simple remove the “z” axis information of the points).

  2. And if I get the .xml for the camera and the projector, should I use it by the opencv just like calibrate a picture?

5

The projector can be treated as the inverse of a pinhole camera model, so we can make use of the same equations to convert between pixel coordinates and 3D coordinates. Given the point [X Y Z] in the camera’s coordinate system solve for [u v] to get the same point in projector pixel coordinates:

image

Further info on this equation here.

That’s correct - the camera parameters from the ProCamCalib .xml should be used for camera calibration, and the projector parameters are used in the above equation.

6

Dear ColtonBarr:

Thank you very much!

Should I build a vtk sense with a cameraMimicProjector to mimic the real projector first? I get the intrinsics and extrinsics of projector and I don’t know how to define the cameraMimicProjector in vkt/slicer?