How to load the image streaming from opencv to 2D view

Development
1

I write a script based on opencv pacakge to capture the video streaming.

As is shown, now I want to show the image on the red slice view ,what should I do?

I know Plus can help,but that is not what I want on my project.

def onCamera(self):
        with slicer.util.tryWithErrorDisplay("Failed to compute results.", waitCursor=True):
    
          if self.ui.camButton.text =="Start":
            self.ui.camButton.text = "Stop"
            self.timer.connect('timeout()', self.ImageShow)
            self.timer.start()
            self.cap = cv2.VideoCapture(0)
          
            self.cap.set(cv2.CAP_PROP_BUFFERSIZE, 3)
            self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, 500)
            self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 500)
            
          else:
            self.ui.camButton.text = "Start"
            self.timer.stop()
            self.cap.release()
def ImageShow(self):
        try:
          if self.cap.isOpened() != 1:
           
            return
        
          ret, img = self.cap.read()
          cv2.imwrite('temp.jpg', img, [cv2.IMWRITE_PNG_BILEVEL, 0])
          Pic = qt.QPixmap('temp.jpg')
          self.ui.imageBox.setPixmap(Pic)
          self.ui.imageBox.setScaledContents(True) 
          
        except Exception as e:
          print('%s' % e)
2

If you can capture the image into a numpy array then you can use that array to update a volume node. See examples in the script repository.

However, in general, I would recommend to acquire images, tracking data, and other sensor data in a separate process and stream it into Slicer using Plus toolkit. This provides more flexibility in what devices you can use and where the data is acuired; and makes the application more responsive (as image acquisition does not block the main thread of the GUI application).

1 Like
3

Thank you Mr Lasso!

I find this function:
slicer.util.loadVolume(ā€œtemp.jpgā€, {ā€œsingleFileā€: False})

And I get the same result with slicer.util.addVolumeFromArray(img)

1 Like
4

The safest is to allocate an image of the required size and scalar type, then get it as a numpy array, and then copy the pixels from the numpy array that OpenCV provides.

It would be cleaner and would probably have perceivably better performance if you acquired the images in a separate process and streamed into Slicer using OpenIGTLink. If you donā€™t want to develop any code then you can use the Plus Toolkit. If you want to develop this yourself then you can write a short Python script that acquires the image using OpenCV and sends it using pyigtl Ā· PyPI.