From @pieper on Wed Aug 08 2018 22:03:18 GMT+0000 (UTC)
Andrey and I did this script to go from dicom to nrrd - it could definitely be generalize. Maybe we want to add some example going to dicom for various common types.
From @fedorov on Wed Aug 08 2018 22:06:27 GMT+0000 (UTC)
Andras, I agree with you, but scripting Slicer functionality in python, and running the result in batch mode has a very big startup overhead, and is associated with other inconveniences, such as tricks that need to be done for headless mode, large package size, platform-specific launcher issues. It might be a lot more convenient for users to use command line tools that are designed for the specific task (when they are available!) instead of learning how to write Slicer scripts, and then figuring other issues.
I am all for Slicer, as you know, but for example using Slicer for batch conversion of DICOM from command line is extremely slow and not convenient, so for that task I would rather use plastimatch, dcm2niix, or dicom2nifti, which are all maintained, and do the job well and much more efficiently.
In this particular case, indeed, if nifti2dicom is not supported (there has not been much activity, and I am not sure about the quality of the code), Slicer might be the only option.
From @lassoan on Wed Aug 08 2018 23:10:16 GMT+0000 (UTC)
I believe that having so many simple, rigid, single-purpose dicomToX tools is bad, because these tools hide issues (by making random assumptions and ignore various potential problems) and cause fragmentation in the research community. It would be better to establish one common DICOM import/export tool framework that many people can use, customize, and extend.
I agree with your list of limitations above (although maybe the recently completed PythonSlicer interpreter solved some of these issues). This common DICOM converter framework does not have to be Slicer-based. However, it should be general, customizable, and extensible, similarly to Slicerās DICOM import/export infrastructure.
From @pieper on Wed Aug 08 2018 23:18:16 GMT+0000 (UTC)
It would be great to factor out the code in slicer so that it could be used in other scenarios in addition to the slicer gui. We already do this for things like the diffusion plugin that uses DWIConvert under the hood or QuantitativeReporting that uses dcmqi.
Regarding your other points Andrey, while they are generally true I donāt see things like package size or startup time as limiting factors. Converting files like this is not a real time activity and if you have a large number of dicom files to convert slicer is the least of your disk space concerns. No reason to prematurely optimize something like this.
From @jcfr on Wed Aug 08 2018 23:18:50 GMT+0000 (UTC)
> , and running the result in batch mode has a very big startup overhead,
We need to address this. (the first step was to lazily load CLIs which was implemented before 4.8.1, now when a specific module is request for batch processing, we need to improve the dependency management and load only the needed dependencies)
and is associated with other inconveniences, such as tricks that need to be done for headless mode,
That is currently a problem indeed.
- for VTK rendering, compiling with mesa offscreen rendering backend would help. Ideally, this should be dynamic and not a compile option.
- for Qt GUI ā¦ this is definitively more tricky was a lot of module do not cleanly separate Widget from logic.
Now, I think that putting effort into having a well maintained docker image supporting this (we could even have one based of ānvidia-dockerā) would address 80% of the batch processing use case.
large package size
Removing EMSegment, BRAINSTools and SimpleITK for the core should help.
platform-specific launcher issues.
Do you have more details ? (if that helps, the PythonSlicer launcher can now be used on all platform including installed macOS)
From @pieper on Thu Aug 09 2018 00:19:58 GMT+0000 (UTC)
> Now, I think that putting effort into having a well maintained docker image supporting this (we could even have one based of ānvidia-dockerā) would address 80% of the batch processing use case.
If it gets the job done Iām okay with a docker based solution, but of course thatās the opposite of lightweight.
I would lean toward trying to make sure all the dicom conversion code is logic-only and could be imported without pulling vtkRendering or Qt into the python interpreter (PythonSlicer for now, ultimately a regular python).
Also, thinking about the actual workflow of converting a large set of nii files to dicom, Iād personally want to do a lot of QC checking because so many things could be inconsistent in the nifti files. So what Iād probably do is start with the CaseIterator and create a custom scripted module to call the dicom export into an organized directory hierarchy (and also into the Slicer dicom database).
From @fedorov on Thu Aug 09 2018 18:37:43 GMT+0000 (UTC)
Great discussion, would be even better if we could have it in-person!
having so many simple, rigid, single-purpose dicomToX tools is bad, because these tools hide issues (by making random assumptions and ignore various potential problems) and cause fragmentation in the research community. It would be better to establish one common DICOM import/export tool framework that many people can use, customize, and extend.
First, I disagree that a general statement like this is accurate. Any tool has issues, and (at least for an outsider) makes random assumptions, and Slicer is no exception, but many of the converters have been around for quite a long time, and are quite robust (arguably, more robust than Slicer!) in dealing with various problems. I think non-Slicer converters from DICOM have larger user base, and may be more reliable just because more people tried and broke them before (e.g., most likely dcm2niix has a much larger user base than Slicer DWConverter).
Second, fragmentation is inevitable, since research community by definition has different interests and focus areas. But that is good! I do not believe in monolithic tools that solve all problems for everyone. Would it be better to establish one common tool? Maybe. But I donāt see how it can be done practically. I wonder if there is any example where something like this has been done successfully in the research community.
platform-specific launcher issues.
Do you have more details ? (if that helps, the PythonSlicer launcher can now be used on all platform including installed macOS)
The lack of console output on Windows is a big one for me, and based on many discussions over the past years.
Regarding your other points Andrey, while they are generally true I donāt see things like package size or startup time as limiting factors. Converting files like this is not a real time activity and if you have a large number of dicom files to convert slicer is the least of your disk space concerns. No reason to prematurely optimize something like this.
I think package size is an issue for someone composing a docker image. I think it is common for folks to try to minimize that, and this converter (~1Gb for Slicer?) would be just a small piece of it. For me, bottom line is using Slicer for converting DICOM in batch mode is cumbersome, but more importantly - we donāt know how robust it is compared to other conversion tools (I am talking here about āfrom DICOMā pathway). What we do know is that the DICOM IO components of ITK need more love, we also know that for sure āassumptions were madeā by people who are no longer around to maintain that code, and we also know that we do not have a robust regression testing for DICOM conversion in Slicer. So all things considered, I am leaning much more towards recommending the converters listed in the project I started last January. Unfortunately, itās been difficult to make time to continue with that project. If and when we establish some technical capability to compare converters across the board on a versatile dataset, that might be the first step towards educated decision making and harmonization of tools.
From @lassoan on Thu Aug 09 2018 20:46:31 GMT+0000 (UTC)
> Any tool has issues, and (at least for an outsider) makes random assumptions
I agree. My point is that based sharing solutions to common problems lowers the development and maintenance workload in the long term (more time is saved than the overhead of coordination between different groups). If you just let everybody do what feels convenient, the easy 90% of the code is reimplemented many times, and the difficult 10% has partial implementation in random toolkits. Eventually, a few well-designed, full-featured solutions emerge as winners, but this process can be accelerated with some conscious effort.
I do not believe in monolithic tools that solve all problems for everyone
I agree, monolithic tools are not well suited for this. The problem exactly is that various groups maintain their own monolithic tools, with zero modularity and potential for reuse or customization. Instead, we should have a small common framework that many groups are using and contributing to - with shared infrastructure components (such as tilted gantry, variable image slice spacing, patient name, etc. management) and application-specific plugins (that can handle import/export of a particular DICOM IOD).
The lack of console output on Windows is a big one for me
Weāve fixed CTK to have a full-featured interaction Python console, even on Windows. It works really well! However, these developments have not been merged yet . @jcfr could you please merge the pull requests and update Slicer master?
6 posts were split to a new topic: Migrating GitHub issue to discourse
From @fedorov on Sat Aug 11 2018 19:04:53 GMT+0000 (UTC)
As I discovered while working on something else, plastimatch convert has a function to create DICOM series from an ITK volume. Looks like it only outputs CT, but it provides options to populate metadata from an existing DICOM dataset.