Orthogonal and Oblique MPR Slicing and Fusion

Hi, I’m Doug Porter from the Mayo Clinic. We have an application that we developed in-house called QREADS which runs in MS Windows and is used by our clinicians. We are looking to expand it’s capabilities from being a simple 2D medical image viewer, to including some 3D functionality. What would be perfect and productive is a simple open source executable program that we could launch from QREADS to which we could pass the path of CT, MR or NM DICOM files, and allow our physicians to perform orthogonal (sagittal, coronal and axial) and oblique MPR triangulation. ALSO, we need a simple UI for volume slice Fusion; co-registration of CT PET MR and SPECT. I understand that this is a Slicer group. But would anyone know how we could acquire such an app? I’ve been modifying a download of FourPanesViewer to meet out needs, but it’s going to require a lot more work. I suspect there is something in Slicer that can be separated out that could receive a DICOM path and perform MPR triangulations. I’ve been in conversation with David Gobbi and he has been really helpful with using his decompression and using CMake, but CMake is very complicated to say the least. I need help from the VTK/ITK experts like yourselves.

If you have MPR Slicer code that I can isolate as a separate app and pass a DICOM path to and branch to from QREADS, that would save us so many headaches…

Thanks,

Doug Porter
Mayo Clinic Software Developer

Hi Doug -

I can’t promise anything will be simple, but certainly a program like you describe is possible using Slicer as a base. Most of the complex infrastructure already exists (dicom parsing, MPR display, linear and nonlinear transforms, annotation tools), and everything you need is exposed for python scripting. It would just be a matter of setting up a startup script that configures the application to meet your needs, no doubt by hiding a lot of unneeded interface elements. Something similar can be done with C++ if that’s a preference, but python is typically more productive given the choice.

Perhaps others on this forum can give you a sense of how long it takes to get up to speed as a newcomer to Slicer and become productive at adapting it.

-Steve

Hi Steve,

Thank you for your response. Sounds like Python is the new love in the industry. So, I need to modify our current Java app called QREADS. To deploy it to 30,000 Mayo Clinic Windows 10 PCs, we simply copy files to the user’s system with no user intervention for installation. In the attached video, I modified a version of FourPanesView and I launch it from QREADS via a dropdown MPR menu item selection, pass it the DICOM file path, and QREADS control returns after the user is done with the UI.

(PLEASE VIEW THE ATTACHED 2 minute VIDEO):

   https://www.dropbox.com/s/okffj0eal4hxn52/QREADS_3D_MPR_Only.mp4?dl=0

Two questions:

  1. Would I still be about to easily deploy to 30,000 workstation?
  2. Would the user experience still be a smooth transition from QREADS to Splicer and back to QREADS?

Hi Doug -

Thanks for the video, that helps. Yes, Slicer could be used to implement that (plus more if it were needed). Slicer works on Windows 10, but also mac and linux if that ever were needed. It also is just a user install (no admin needed). The main downside I can think of is that the Slicer installation is pretty big if all you use is the MPR modes.

Not sure about the transition back and forth. They would be two applications so maybe some work would be needed by the user or you could automate something at the system level.

Perhaps more of a departure from your current implementation but you might also consider using the web-based OHIF as a platform. It also supports MPR and slab rendering, and is zero-install (just runs in browser, as long as it’s fairly recent). You also need to figure out how best to serve the data for that.

Here’s an example you can test. Click the “2D MPR” tool to experiment with that.

https://viewer.imaging.datacommons.cancer.gov/viewer/1.3.6.1.4.1.14519.5.2.1.6279.6001.224985459390356936417021464571?seriesInstanceUID=1.2.276.0.7230010.3.1.3.0.57823.1553343864.578877,1.3.6.1.4.1.14519.5.2.1.6279.6001.273525289046256012743471155680

So there are several good options for you to consider.

-Steve

Could I use a browser object within my app inside of a window or java composite?

Hmm, maybe. I don’t know much about current Java. But you could open a browser tab I guess.

If install size is an issue then you can create a minimal build of the application, which only contains components that you need. Its size would be probably about 500-600MB (200-250MB zip file). With some work, we could reduce number of included VTK packages, remove some more dependencies, shaving off potentially an additional 100-200MB.

If you need to pass data between Slicer and your external application then Slicer can open a web server interface, so that you can remote control it and collect data using REST API.

What is the role of QREADS? Does it just connect to PACS and query/retrieves data? Or has other specialized functions?

Oh Hi Andras! We can communicate here. QREADS was first written back in the 90’s when DICOM implementation was fairly new. It is client server based and very scalable. The server does al of the the work and maintains short term image storage on file servers while archiving into long term using DICOM communication to store and retrieve. The QREADS server is very scalable and can handle Mayo on the enterprise level for all of the sites around the country. The QREADS server streams images down to a folder on the local QREADS client viewer workstation of which there are around 30,000 in quantity. It’s surprisingly fast for the load it has to handle and other vendors are much too expensive to replace this infrastructure. The major complaint about QREADS is that MPR and Fusion are not supported and that’s my quest. The demo is my prototype to the docs but it was a pretty tedious endeavor. I am 66 years old and this is my last hurrah before retirement which is becoming more and more tempting. Needless to say, I don’t want to spend the next 5 years reinventing the wheel :slight_smile:

Hey Steve, could the browser code be downloaded and executed locally to access a local folder? Or does it implement DICOM query/retrieves? How did you load the data in the Browser?

Nowadays, basic viewing features (including MPR and fusion) are expected to be available in the web browser (and they are available, for free, for example in OHIF viewer). For more advanced annotation/segmentation/analysis you would launch another web application or a local application (such as 3D Slicer) from the web browser.

If QREADS already uses DICOM PACS servers for storage and retrieval with standard DIMSE or DICOMweb interface then OHIF and Slicer should be able to connect directly to them already, query what is available on the server, show patient/study/series list, retrieve and display what the user needs, and even edit/annotate data, and push it back to the DICOM server. Probably some enhancements are needed to reach full feature parity with your existing solution (e.g., provide advanced query options, make sure performance is good even if you have tens of millions of studies, configure user authentication/access control, …), but this would not be a tremendously big effort and there would be synergies with various projects, so you would not need to do it all by yourself.

Ok, of all of the 30,000 standard Mayo PCs my development one had somehow been down graded to OpenGL 1.0. I didn’t suspect this because it worked 2 weeks ago. Anyway, sorry for the waste of time and stress. Slicer is good now. Now I can try some of your suggestions. Thank you so much.

1 Like

Do you have an NVidia GPU in your developer computer?

No it’s an Intel card, but it works now. So, I’m good. I tried the script you suggested. See below…

“C:\ProgramData\NA-MIC\Slicer 4.11.20200930\Slicer.exe” --no-splash --python-code “setToolbarsVisible(False); setMenuBarsVisible(False); slicer.app.processEvents(); mainWindow().findChildren(‘QDockWidget’,‘PanelDockWidget’)[0].hide(); slicer.app.processEvents(); loadVolume(r’H:\VTKTest\Images4VTK\CTChestThinSlice’)”

Slicer comes up fine. I’m specifying a folder to the loadVolume function, not a file. Also the images are compressed DICOM. No images load though. What am I doing wrong?

Thanks,

Doug

There are a couple of ways to load DICOM data into Slicer. The method that I described requires a filename as inputs and that all data in the series are stored in that same folder. How to launch Slicer from command line to view a DICOM data set is a slightly different from the scope of this discussion, so it would be great if you could post it as a new topic.

OK Thanks! I’ll try that single file thing now.

OK, I obviously lack the knowledge and experience with Slicer that I need in order to solve my problems without being spoon-fed by y’all experts. How do I catch up? Are there good beginner tutorials out there that would teach me how to do what I need to do? Thanks.

Yes, it can be hard to learn.

Did you find the developer resources here? Were they helpful?

PerkLab’s Slicer bootcamp training material is a good start, especially the Scripting and module development tutorial. You can also check out SlicerMorph tutorials.

It is not a problem if you ask here anything - next time someone will have the same question will find it on Google, so it is a good investment of our time.

Thanks Steve, I’ll go there to read.

OK, I’ll consider this as a slicer-documenting forum.

So where were we? So here is what I get when I issue the following command. Just one frame not orthogonal slices generated and I can’t scroll through the axials.

“C:\ProgramData\NA-MIC\Slicer 4.11.20200930\Slicer.exe” --no-splash --python-code “setToolbarsVisible(False); setMenuBarsVisible(False); slicer.app.processEvents(); mainWindow().findChildren(‘QDockWidget’,‘PanelDockWidget’)[0].hide(); slicer.app.processEvents(); loadVolume(r’H:\VTKTest\Images4VTK\CTChestThinSlice’)”