# How to capture a screenshot and turn it into a numpy array with python code?

**URL:** <https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302>\
**Category:** Support\
**Tags:** python\
**Created:** [March 4, 2022, 2:56am UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302 "2022-03-04T02:56:00Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![user4](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/user4/32/13172_2.png) [@user4](https://discourse.slicer.org/u/user4)\
**Post date:** [March 4, 2022, 2:56am UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/1 "2022-03-04T02:56:00Z")

</div>

Slicer version 4.11  
I can capture a screenshot here:  
 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/3/b/3b20a94b9e54d0137ac5e713b1f0492d50a23c15.png)  
or I can capture a screenshot in Script Repository [here](https://slicer.readthedocs.io/en/latest/developer_guide/script_repository.html#screen-capture)

However,both of the two methods are saving the image as .png file.If I want to see the image pixel value I have to load this image file.  
So there is some way to capture a screenshot in python and make it a numpy array without saving it?

Thank you advance for your help and advice!!

---

<div class="post-metadata">

**Author:** ![jamesobutler](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jamesobutler/32/7511_2.png) [@jamesobutler](https://discourse.slicer.org/u/jamesobutler)\
**Post date:** [March 4, 2022, 3:10am UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/2 "2022-03-04T03:10:13Z")

</div>

To get a slice of a volume as a numpy array you can use some of the examples such as

[https://slicer.readthedocs.io/en/latest/developer\_guide/script\_repository.html#get-axial-slice-as-numpy-array](https://slicer.readthedocs.io/en/latest/developer_guide/script_repository.html#get-axial-slice-as-numpy-array)

---

<div class="post-metadata">

**Author:** ![user4](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/user4/32/13172_2.png) [@user4](https://discourse.slicer.org/u/user4)\
**Post date:** [March 4, 2022, 7:08am UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/3 "2022-03-04T07:08:29Z")

</div>

Thank you james but this is not what I need,  
I have to use the screenshots image cause it is after ray casting.

```auto
# Capture RGBA image
renderWindow = view.renderWindow()
renderWindow.SetAlphaBitPlanes(1)
wti = vtk.vtkWindowToImageFilter()
wti.SetInputBufferTypeToRGBA()
wti.SetInput(renderWindow)
writer = vtk.vtkPNGWriter()
writer.SetFileName("c:/tmp/screenshot.png")
writer.SetInputConnection(wti.GetOutputPort())
writer.Write()

```

I think there must be an interface to get the image in buffer,however I am not sure how to get the data.

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [March 4, 2022, 2:36pm UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/4 "2022-03-04T14:36:29Z")

</div>

This example [`vtkImageDataToPNG`](https://github.com/pieper/SlicerWeb/blob/master/WebServer/WebServer.py#L414-L426) may help. It’ll give you an array of the png compressed image by you can get something similar if you just use the `vtkWindowToImageFilter` output directly.

---

<div class="post-metadata">

**Author:** ![user4](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/user4/32/13172_2.png) [@user4](https://discourse.slicer.org/u/user4)\
**Post date:** [March 5, 2022, 5:52pm UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/5 "2022-03-05T17:52:58Z")

</div>

Thanks Steve! You really gave me some insight! I think I am close to the solution but maybe need a little more help!  
I saw the example and did it in three steps:

1. Get the render window

```auto
view = slicer.app.layoutManager().threeDWidget(0).threeDView()
renderWindow = view.renderWindow()
renderWindow.SetAlphaBitPlanes(1)

```

1. Use the filter to get vtk data

```auto
wti = vtk.vtkWindowToImageFilter()
wti.SetInputBufferTypeToRGBA()
wti.SetInput(renderWindow)
# wti.ReadFrontBufferOff()
wti.Update()
vtk_data = wti.GetOutput()

```

1. Use _vtkImageDataToPNG_ function to get pngArray

```auto
def vtkImageDataToPNG(imageData):
    """
    Return pngArray using the data from the vtkImageData.
    """
    writer = vtk.vtkPNGWriter()
    writer.SetWriteToMemory(True)
    writer.SetInputData(imageData)
    # use compression 0 since data transfer is faster than compressing
    writer.SetCompressionLevel(0)
    writer.Write()
    result = writer.GetResult()
    pngArray = vtk.util.numpy_support.vtk_to_numpy(result)

    return pngArray

pngArray = vtkImageDataToPNG(vtk_data)

```

After the above three steps,I checked the type and shape of the pngArray as follows:

```auto
type(pngArray)
--> <class 'numpy.ndarray'>

pngArray.shape
--> (3721789,)

```

I have two question for the above results:

1. It seems the array has been flattened,so my next quesion is how to reshape the array to the render window size, in other words,how to get the render window size (height,width).

2. I compared the dimension of the pngArray with the screenshot produced by GUI:  

```auto
import matplotlib.image as mpimg
img = mpimg.imread('test.png')
img.shape
--> (925, 1004, 4)

925 * 1004 * 4
--> 3714800

```

It is strange that the two images are not the same size,in other words,the flattened numbers are different. (3714800 v.s. 3721789).Did something go wrong during the procedure of producing the pngArray?

Thank you for your continued attention and help again!

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [March 5, 2022, 6:34pm UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/6 "2022-03-05T18:34:41Z")

</div>

Hi @user4 -

Yes, you are very close - the example I pointed to was for getting the compressed png data as an array (e.g. the data in a .png file). Instead you need to get the array from the `vtkImageData` that comes from the `vtkWindowToImageFilter`.

Here’s the snippet:

```auto
view = slicer.app.layoutManager().threeDWidget(0).threeDView()
renderWindow = view.renderWindow()
renderWindow.SetAlphaBitPlanes(1)
wti = vtk.vtkWindowToImageFilter()
wti.SetInputBufferTypeToRGBA()
wti.SetInput(renderWindow)
wti.Update()
image_array = vtk.util.numpy_support.vtk_to_numpy(vtk_data.GetPointData().GetScalars())
image_array = image_array.reshape((view.height, view.width, 4))

```

You may need to swap width and height - I didn’t check that for sure.

---

<div class="post-metadata">

**Author:** ![user4](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/user4/32/13172_2.png) [@user4](https://discourse.slicer.org/u/user4)\
**Post date:** [March 6, 2022, 8:10am UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/7 "2022-03-06T08:10:09Z")

</div>

Thanks Steve! You are quite right,with your code above I have get the image array:

```auto
image_array.shape
--> (925, 1136, 4)

```

While I took a screenshot with GUI, and saved the png image file marked in red:

 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/3/d/3d9a5dfe0d62ecbf7f7504e0fea185ae99fbe8b3.png)

```auto
from skimage import io
img = io.imread('test.png')
img.shape

--> (925, 1136, 4)

```

yeah, It seems okay because the dimensions of both images are the same.  
However,when I check the value of array it is not the same,am I missing something or maybe forgot some parameters setting?

```auto
(img == image_array).all()
--> False

```

P.S  
This image is produced by image array with your help and code:  
 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/6/e/6e0270d3da805b8c3f6b68b896e4f0895d7da8d0.png)

This image is produced by screenshot capture GUI:  
 ![image](https://us1.discourse-cdn.com/flex002/uploads/slicer/original/3X/2/4/242a1b224d00c93e6c469664fef3a3d506f53a5a.png)

Apparently,the two images are different from each other,do you think I am missing some parameters?

---

<div class="post-metadata">

**Author:** ![pieper](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/pieper/32/8_2.png) [@pieper](https://discourse.slicer.org/u/pieper)\
**Post date:** [March 6, 2022, 1:56pm UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/8 "2022-03-06T13:56:34Z")

</div>

Ah yes, I forgot that detail. It would be easier to tell if you load some data but you can see it in the gradient too: the images are mirrored vertically from each other. This is because vtk image rows go from bottom to top (like xy coordinates of a graph), while most image formats stack the rows from top to bottom (like scan lines of an old tv). So you need to rearrange the rows in memory to make the match.

---

<div class="post-metadata">

**Author:** ![user4](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/user4/32/13172_2.png) [@user4](https://discourse.slicer.org/u/user4)\
**Post date:** [March 6, 2022, 2:53pm UTC](https://discourse.slicer.org/t/how-to-capture-a-screenshot-and-turn-it-into-a-numpy-array-with-python-code/22302/9 "2022-03-06T14:53:48Z")

</div>

Thanks a lot Steve! 👍I have solved the problem just reversing the image array.  
However it can not be done without your help,I am really grateful to your help again!