How to unset default 'compress' option during save

Hi,

Is there a way to default compress option to NO for save dialog? Compress option is not visible immediately to new Slicer users, and compressing single threaded such large dataset (2000^3) is extremely slow. We are talking about 15 sec for uncomressed data and dozens of minutes with compress enabled on an ssd. Again for a new user, it is not immediately clear to the user whether Slicer stalled/crashed or doing something.

In case someone wants to see, here is a test dataset:
https://ndownloader.figshare.com/files/12808130

In the Save Dialog box, click on “Show options”, then you can toggle off “Compress” option in the options column.

-Andinet

For some users it makes more sense to enable compression by default, for some others, disabling compression by default is preferable.

You can set default file type and saving options by specifying a default storage node by adding 3 lines to your application startup script as shown in the script repository.

To me it’s easier to learn to set the save option than it is to add a .slicerrc.py option.

But rather than making this either compression or fast save, maybe we can try to get the benefits of both with something like this: https://zlib.net/pigz/ (I haven’t tried this).

From a user point of view, this would make sense to be a setting under preferences option.

I use pigz from command line, and it works really well on multi core systems. It would be good to have it in slicer.

Yes, I can see how this would be a convenience for people who always want one thing or the other. My concern is that this is pretty hidden away and is pretty obscure. With nrrd files it’s not easy for users to know if it’s compressed (need to look in the header or guess based on file size). So sticking with compressed all the time is the best thing if we can make it fast enough.

Just to confirm @muratmaga would you really want lots of 2000^3 files on your disk that were mostly zeros?

Follow up on pigz, there’s a github repo and the author seems quite legit. But there’s no obvious windows support so probably a cmakeification effort would be needed.

Even SSD storage is so cheap now that the time consumption waiting for lots of zeros to be compressed into fewer zeros is hardly justifiable, IMO. People working with large volumes are very likely to be working with fewer datasets than people working on medical analyses, and more likely to use the software interactively doing bunch of one of things that will require them to save repeatedly derivative datasets.

I say +1 for the pigz implementation. Or some indication of progress of saving/compression process. Otherwise Slicer simply looks stalled.

Another suggestion I have is to make compress visible field visible next to the format in the default save dialog box (as oppose to requiring user to click on options to expand the options). That can be good compromise.

@lassoan I am not finding a compress example. What would be option to specify for compress?

Yes, if we think that’s a commonly used option it could make a lot of sense to make it visible by default.

You can enable/disable compression using storage node’s SetUseCompression method.

Thank you. Based on the example, this works:

#set the default 3D model format to PLY (instead of VTK)
defaultModelStorageNode = slicer.vtkMRMLModelStorageNode()
defaultModelStorageNode.SetDefaultWriteFileExtension(‘ply’)
#to turn off compression for the models
defaultModelStorageNode.SetUseCompression(0)
slicer.mrmlScene.AddDefaultNode(defaultModelStorageNode)

But I couldn’t figure out how to use the provided Change file type for saving for all volumes (with already existing storage nodes) example to work with fresh scenes with no volume loaded.

defaultVolumeNode = slicer.vtkMRMLScalarVolumeNode()
defaultVolumeNode.SetDefaultWriteFileExtension(‘mha’)

fails with AttributeError: ‘vtkCommonCorePython.vtkMRMLScalarVolumeNode’ object has no attribute ‘SetDefaultWriteFileExtension’

You need to change the default volume storage node (not the default volume node).

Thanks, but looks like I need more guidance…

defaultStorageNode = slicer.vtkMRMLStorageNode()

fails with TypeError: this is an abstract class and cannot be instantiated

and I can’t locate a vtkMRMLVolumeStorageNode() (or rather how/where to find it).

Here’s an example:

#set the default volume storage to not compress by default
defaultVolumeStorageNode = slicer.vtkMRMLSegmentationStorageNode()
defaultVolumeStorageNode.SetUseCompression(0)
slicer.mrmlScene.AddDefaultNode(defaultVolumeStorageNode)
logging.info("Segmentation nodes will be stored uncompressed 

This is also in theScript Repository now.

This came up in another thread recently:

We could potentially get a big performance boost at minimal file size cost by changing the default gz compression level to 1. In some small-scale tests I did a few months ago, there was a major time cost between high and low settings, for only a tiny compression improvement (e.g. 90-> 26 MB at level 9 in 34s, vs 90->27 MB at level 1 in <1s). This was only a small test on a few brain MR datasets, but it seems reasonable.

Using a low compression level would make a lot of sense, as one of the main reasons I would keep compression enabled is to deflate blank voxels around images that have a non-rectangular shape.

Is there an API to change compression level in ITK IO classes?

Looks like only for a few specific formats (MINC, PNG). For NRRD we can set it in teem directly.

@muratmaga tomorrow’s nightly build will have this commit which changes the default nrrd compression to ‘low’ so it should be faster and still pretty good compression. Let us know how it works in your use cases.

https://github.com/Slicer/Slicer/commit/d87fdfd3eeca502c97cf611c92e443571e569292