ChatGPT knows a little about Slicer

I tried to ask ChatGPT about 3D Slicer. Here is it’s frighteningly good responses. In some (most?) cases, the responses are convincing but wrong. In general, ChatGPT is very good at writing code, though. I include several questions and answers here:

Q: How do I write an extension module for 3D Slicer?

A: To write an extension module for 3D Slicer, follow these steps:

  1. Install 3D Slicer on your computer.
  2. Create a new folder in the “Modules” directory in the 3D Slicer installation folder. This will be the directory for your extension module.
  3. In your extension module directory, create a file called “CMakeLists.txt” and include the following code:
slicerMacroBuildScriptedModule(
  NAME MyModule
  SCRIPT_FILENAME MyModule.py
  RESOURCES MyModule.png
)
  1. Create a Python script called “MyModule.py” in your extension module directory. This script will contain the code for your extension module.
  2. In the “MyModule.py” script, define a class that subclasses “vtk.vtkMRMLAbstractLogic”. This class will contain the logic and functionality of your extension module.
  3. In the class definition, define a method called “processMRMLNodesEvents(self, caller, event, callData)” which will be called whenever a MRML node is modified in 3D Slicer.
  4. Define any additional methods or functions that you want your extension module to have.
  5. In the “MyModule.py” script, create an instance of your class and call the “register()” method on it to register the extension module with 3D Slicer.
  6. In the “Modules” directory, run the “build_all.py” script to build your extension module.
  7. In 3D Slicer, go to the “Modules” menu and select “MyModule” to use your extension module.

Q: How do I create a 3D Slicer module using Qt and Python?

A: To create a 3D Slicer module using Qt and Python, follow these steps:

  1. Install 3D Slicer on your computer and make sure that Qt is included in the installation.
  2. Create a new folder in the “Modules” directory in the 3D Slicer installation folder. This will be the directory for your module.
  3. In your module directory, create a file called “CMakeLists.txt” and include the following code:
slicerMacroBuildScriptedModule(
  NAME MyModule
  SCRIPT_FILENAME MyModule.py
  RESOURCES MyModule.png
  WITH_PYTHONQT
)
  1. Create a Python script called “MyModule.py” in your module directory. This script will contain the code for your module.
  2. In the “MyModule.py” script, import the necessary modules from Qt and Slicer:
from PythonQt import QtCore, QtGui
import slicer
  1. Define a class that subclasses “slicer.qSlicerAbstractModuleWidget” and “QtGui.QWidget”. This class will contain the GUI and functionality of your module.
  2. In the class definition, override the “setup()” method to set up the GUI for your module. Use Qt designer to design the GUI and then use the “loadUi()” method to load the GUI into your module.
  3. Define any additional methods or functions that you want your module to have.
  4. In the “MyModule.py” script, create an instance of your class and call the “register()” method on it to register the module with 3D Slicer.
  5. In the “Modules” directory, run the “build_all.py” script to build your module.
  6. In 3D Slicer, go to the “Modules” menu and select “MyModule” to use your module.

Q: How about a C++ Slicer extension using Qt?

A: To create a 3D Slicer extension using C++ and Qt, follow these steps:

  1. Install 3D Slicer on your computer and make sure that Qt is included in the installation.
  2. Create a new folder in the “Modules” directory in the 3D Slicer installation folder. This will be the directory for your extension.
  3. In your extension directory, create a file called “CMakeLists.txt” and include the following code:
slicerMacroBuildModule(
  NAME MyModule
  SRCS MyModule.cxx
  MOC_SRCS MyModule.h
  RESOURCES MyModule.qrc
)
  1. Create a C++ file called “MyModule.cxx” in your extension directory. This file will contain the code for your extension.
  2. In the “MyModule.cxx” file, include the necessary Qt and Slicer headers:
#include <QtGui>
#include <vtkSlicerConfigure.h>
#include "MyModule.h"
  1. Define a class that subclasses “vtkSlicerModuleLogic” and “QObject”. This class will contain the logic and functionality of your extension.
  2. In the class definition, override the “SetMRMLSceneInternal(vtkMRMLScene* newScene)” method to set up the extension when it is added to the MRML scene.
  3. Define any additional methods or functions that you want your extension to have.
  4. In the “MyModule.cxx” file, create an instance of your class and call the “Register()” method on it to register the extension with 3D Slicer.
  5. In the “Modules” directory, run the “build_all.py” script to build your extension.
  6. In 3D Slicer, go to the “Modules” menu and select “MyModule” to use your extension.

Q: how do I volume render a nifti file containing CT data in 3D Slicer?

A: To volume render a NIFTI file containing CT data in 3D Slicer, follow these steps:

  1. Load the NIFTI file into 3D Slicer by going to the “File” menu and selecting “Add Data”.
  2. In the “Modules” menu, select “Volume Rendering” to open the volume rendering module.
  3. In the “Volume Rendering” module, select the NIFTI file in the “Input Volume” dropdown menu.
  4. Click the “Apply” button to display the volume in the 3D view.
  5. In the “Volume Rendering” module, adjust the rendering settings as desired, such as the color map, threshold, and opacity.
  6. Click the “Apply” button to update the volume rendering with the new settings.
  7. To save the volume rendering, go to the “File” menu and select “Save Screenshot”. Choose a file name and location, and select “Volume” in the “What to Save” dropdown menu. Click “Save” to save the volume rendering.

Q: What is OpenAnatomy?

[oops! ChatGPT is confidently wrong!]
OpenAnatomy is a software platform that enables users to view, explore, and analyze 3D anatomy data. It is designed to facilitate the development and dissemination of open anatomy resources, including segmented models of organs, bones, and other structures. OpenAnatomy can be used in a variety of settings, including education, research, and clinical practice. It is developed and maintained by the BioMedical Informatics Research Network (BIRN) and is open source, meaning that users can access and modify the source code.

Q: Can you describe the DICOM coordinate system?

[I believe this isn’t correct.]
The DICOM (Digital Imaging and Communications in Medicine) coordinate system is a standardized system used to represent the location of points and objects in 3D space. It is used in medical imaging to describe the position and orientation of images, as well as the position of structures within the images.

In the DICOM coordinate system, the origin is located at the center of the imaging volume. The x-axis points to the right [is this subject right or observer right?], the y-axis points anteriorly (towards the front of the body) [I believe that’s wrong], and the z-axis points superiorly (towards the head). The orientation of the axes is determined by the orientation of the patient’s body during the imaging process.

The DICOM coordinate system is a right-handed coordinate system, meaning that if the x-axis, y-axis, and z-axis are visualized as three perpendicular lines, then the direction in which the lines form a positive rotation (like the hands on a clock) is considered “right-handed”.

In the DICOM coordinate system, the unit of measure is the millimeter (mm). The dimensions of the imaging volume are also specified in millimeters.

2 Likes

The instructions ChatGPT gave for Slicer are also wrong too haha. They are snippets of things you might do in various places, but put together don’t make sense.

To start developing a C++ module you’re not going to install Slicer, but rather build Slicer from source. Creating a new folder in the modules directory is also not going to work out well as described.

ChatGPT using GPT-3.5 model which has info up to Q4 2021 has some snippets of info about Slicer, but nothing that a new person could use. An experienced developer could filter out the junk, but unclear how much time it would save at this point.

What it can do pretty well is help with code that has a really good documented API doc which it knows about from its learning. From my experience if you ask it a basic task to create a function to do something with say numpy, it can do it fairly well.

Yes, I asked it to write some code for a Babylon.js and React integration, for instance, and the answer was basically right on target. It’s pretty neat that it even attempts to do anything with 3D Slicer. Maybe that’s its biggest weakness as well: it’s eager to please, and doesn’t know when to say “I don’t know” or “I’m not sure”.

I agree, a big limitation that it provides correct and completely wrong information with the exact same confidence. I think Google search is still more effective for finding usage instructions for Slicer, but I would expect that the same technology could work very well if we could restrict it to mine information only from the current Slicer user manual (and maybe recent Slicer forum posts) and make it a bit less confident in answering questions that it doesn’t actually know the answers for.

Students in our lab use ChatGPT to generate Python code for Slicer and they say it is already useful for them. The provided code rarely runs correctly as is, but it often contains the necessary methods that they can use. It kind of automates the process of typing the question in Google search, clicking on the top 10 hits, and finding the commonly used coding patterns.

1 Like

Yes, it would be nice if it combined the code suggestions with links back to the source so that people could learn the context and not just be stuck with a wrong answer.

User interface questions may be even better than programming questions. I asked “how do I make a segmentation in 3D slicer show as a thick outline with no fill?”, a question which comes up from time to time here on discourse, and I got an answer that is kind of incorrect but maybe useful. I used google translate to try the same question in Spanish and French and the answers in those languages were similarly potentially helpful as far as I could tell.

It’s easy enough to use and if people find it worth the time investment I guess we’ll just see where that takes us.

I was just curious and typed one of the recent forum questions into chatgpt and it gave remarkably accurate answer:

It lacks detailed reasoning and some nuances/exceptions, so it is not ready to completely replace us here, but I find it really impressive.

1 Like

Asking more specific question provided some more details, which were also very accurate and comprehensive:

It is clear, short, I don’t find anything important is missed, it is ordered correctly - starting with the most common/important.

1 Like

I note ‘Raw images are generally not given to patients’.

If this is generally the case in North America, I would be sorry for private practitioners. They would request an exam and have processed images, and not being able to further analyze the exam so as to better fit their reasoning and hence choose between therapeutic options. I’m speaking here of surgery. Without the CT DICOM volumes that patients hand over to me on DVD, I would not have been here typing this, and many patients would have been very sorry for many reasons.

This is not to discredit ChatGPT. Just an incremental comment. :slight_smile:

Unfortunately, ChatGPT is right. In North America, the original images are often retained only for a short time and only the poorer quality processed images are archived (and made available for patients on request).

This is clearly a mistake and we try to make clinicians and hospital IT staff aware of this issue, but it is hard to change this practice, because 3D review is not the standard for most cases, and for 2D slice-by-slice review the processed images appear to be sufficient.

1 Like