Hi. I’m currently using Slicer 5.2.2 to do quite a lot data manipulation using python scripts, and I am not surprised that some of these tasks take a while. However, I am surprised how much slower 5.2.2 is at opening DICOM data compared with 4.11.
I’ve already built the DICOM database, so they are being loaded from that, and not by Add Data or anything. It literally takes 5-10 minutes for Slicer to “check” a variety of plugins, and then a few minutes more to actually load the data. 4.11 used to zoom through the checks within a few seconds, and then load the DICOM data itself in a minute or so.
The only other difference between the systems is 4.11 was running on Ubuntu on a virtual machine on my laptop, and 5.2.2 is running on the same virtual machine but on a server now with twice the RAM and three times the cores. If anything, I would have expected it to be faster just from the increased server power.
I’ve looked at the log messages and nothing obvious jumps out, but I can’t find a way to copy them all and post them in this message. Any suggestions?
Thanks for reporting this. We want dicom loading to be as fast as possible, but perhaps a regression crept in. Can you see what examine steps seem to be taking a long time in the progress dialog? Can you try turning off some of the plugins (options in the lower left of the dicom module) and see if that helps. It would be good if you could compare the different slicer versions on the same machine with the same data.
Sorry for the delay in getting back to you, it’s been a hectic few days. It seems to sit on Checking DICOMPETSUVPlugin at 27% for quite a while but even after that is sits on quite a few other plugins for a while. I’ve disable all plugins and it takes a while to get through DICOMenhance, ImageSequence etc. but maybe not as slow as the extensions. Unticking all of the plugins in the bottom left doesn’t seem to load the data, with nothing showing in the loaded data window nor viewing windows, although the progress bar doesn’t show either. Interesting, CPU usage actually seems pretty low regardless of which of the tests I was running.
I’ve uploaded a screenshot of the log when opening up a PET/CT/RTSTRUCT dataset with only the native DICOM plugins installed, all other extensions disabled (although I think I need an extension for the RTSTRUCT file when doing this for real). It actually took a couple more minutes than the last entry on the log shown here.
As a PET dataset, this report may be what you observed @pieper and reported at the following issue linked below.
It appears development on QIICR repos has ceased? From a user perspective it would be nice to be informed in the Slicer extensions index which extensions are actually being maintained or the last time they were updated as a metric for that.
That’s an interesting idea. For comparison, I have around 100 studies, with between 4 and 25 series each (some series are edited duplicates). I’m only loaded the same original data each time though (the RTSTRUCT, the CT and the PET data).
Just to check, I opened a second instance of Slicer and tried to load just the RTSTRUCT and CT, but it didn’t seem to be any quicker. System resources were barely dented, so having both open at the same time hopefully wouldn’t have affected the test too much.
I’ve added a log below for when loading the PET/CT/RTSTRUCT with extensions etc. I’ll be honest, 50 minutes is longer than I expected, I’m sure previously it was 20-30.
How many patients/studies/series are you selecting at once? When you select a lot then some plugins try to make sense of the complete selection as a whole, which may take a long time.
Have you installed some more extensions after you imported a large data set? That can hugely increase the first examination time (checking of how the series can be loaded) because the new extensions may declare new DICOM tags, therefore all the DICOM files must be parsed again.
If you tell us what data set you are working on and what extensions you have installed then we can have a look if some performance regression has crept in.
Just the one study, with three series (RTSTRUCT, CT and PET). There are around 100 studies/patients in the database, all personally anonymised directly from a couple of Siemens PET scanners.
I’m pretty sure I installed the extensions after building the database though, as I forgot I needed some of them when moving from one setup to another. All of the patients have been opened at least once now though, and reopening the same patient doesn’t speed it up
Unfortunately, I left it building the database for 4.11 overnight, but when I’ve come back to the vmachine this morning, I can’t get into it as my ubuntu version has suddenly started prompting for a password and the expected password isn’t working.
Actually, whilst killing my system trying to build the 4.11 database, I realised that my 5.2 database was actually stored in a networked location, albeit a very fast internal network connection, due to local space constraints on the vmachine. I’ll pump up the partition size and rebuild 5.2, then test and report back.
@Markb can you actually test with Slicer 5.4.0? That is the latest stable version of the code. Slicer 5.2.x is technically unsupported now with no future updates planned for it.
I’ve downloaded 5.4 and pointed it at the same database, and it seemed to recognise the patients straightaway. However, when loading the data, it said it couldn’t load the CT data. I haven’t added any of the extensions as yet, I just wanted to see what the raw time “out of the box” was, and as the image below shows, it was only a couple of minutes. Not sure if the non-loading of the CT slowed or sped that up though or whether there is something related that causes 5.2 to keep attempting something etc.
Adding the extensions definitely made a difference. Still an issue loading the CT but as the image below shows, nearly 15 minutes on a single step at the beginning
Extensions I have are: DCMQI, PETDICOMExtension, PetSpectAnalysis, SlicerDevelopmentToolbox and SlicerRT (not sure if all are needed anymore to be fair, some might be when I was creating python scripts for automating certain tasks)
In looking at the code I also found another issue that may slow things down:
I haven’t worked on PET/CT in a while but I plan to in the coming months so I may get to these, or at least add some profiling to narrow down where the time is being spent. If anyone has a chance to work on it sooner that would be great.
