While volume sizes of the medical imaging (CT/MR) datasets haven’t changed in much in almost a decade, research imaging methods such as tiled microscopy, synchrotron, and nanoCT is pushing larger and larger datasets, any where from 10s of gigavoxels to terabyte of voxels. Even with common desktop microCT scanner, biology labs routinely generate 10GB+ volumes. I only see this trend getting stronger with time.
As the leading open-source biomedical visualization suite, I would like to see better support for such datasets in 3D Slicer. Our typical advise to downsample or crop to specific region is not always helpful, as one reason to create such large datasets in first place is to visualize/analyze entire specimen, not a specific region. Another reason is that structures of interests can only be a few voxel thick, but scattered across the entire volume (for inspiration see this tweet for 40GB stack of grasshopper wings scanned in Lawrence Livermore synchrotron).
I would like to initiate a shopping list of features that will make Slicer more efficient working with such large dataset, and identify ways/funding to incorporate this into Slicer. One assumption I have that users will be using desktops dozens of cores, and powerful GPUs and large RAM, but not necessarily clusters or parallel infrastructures common at supercomputing centers.
My minimum list of features consists of:
- Rapid I/O: parallel read/writes, efficient multi-threaded compression (as importing a 10GB stack with 2000-3000 slicer into Slicer right now takes in the order of minutes. Writing a compressed nrrd of this takes tens of minutes).
- Automatically down sample data to volume render within GPU HW capability (currently an invalid texture dimension error is thrown, if a single texture dimension exceeds the capability of the HW, and don’t render anything).
- Being able to display regional data in full resolution
- Segmentation tools and other working as if this is a normal sized data (e.g., support for OOM execution of datasets).
Feel free to contribute your asks and inputs to this thread.
@pieper @jcfr @thewtex @lassoan
(there was a previous thread that I accidentally delete now is removed. So I started a new one).