Opening file consumes large amount of memory

Hello,
I am so sorry for continuously asking similar questions, but I can’t get my head around this.
I am still suffering from a bad allocation problem.
This time, not during any process but just opening the file use memory up to 63GB.
The memory usage spikes up to 63GB then lowers down to 18GB.
And continuously use about 28GB of my memory without any process.
화면 캡처 2021-08-30 170018
Other files use up to 30-40GB during the opening process.
below is my sample .mrb file.

Can anyone please tell me what I am doing wrong again?
My computer is Window10, 64GB RAM, i7, RTX 3070 Graphic.
Any help would be great.
Thank you always.

While not resolving your problem, I can at least confirm running in the same issue while loading your file on Linux. It consumes 64GB + 16 GB swap space before crashing Slicer. Please update if you find a solution or explanation.

1 Like

What kind of file is this? I’m not in a place where I can test at the moment, but if this is a highly compressed file it might actually require that much memory.

Hello,

I used a series of reconstructed BMP files.
(I don’t remember if it was 16 or 32 bit, but from what I know the dimension of each .bmp file is 1008*1008)

To write down the whole process:

  1. I used Skyscan1272 to take CT and then reconstructed the images into BMP files with a higher bit.
  2. Then I dragged the BMP files into the 3D Slicer using the ImageStacks module.
    a. The Image spacing was set to 0.008um(the pixel size)
    b. I selected half resolution to optimize the process.
  3. Then a series of segmentation processes was done in order to separate my samples from the surroundings.
  4. I recreated another volume with full resolution and then used my segments to create cleaned-up volume of my samples.
  5. I deleted all the volumes except for the cleaned-up version.
  6. Then again, I went through series of segmentation process to separate each samples.
  7. There are 8 samples and I used each segmented area to create new 8 volumes.
  8. Each volume was going to be segmented once again to pull out final data such as volume, area, thickness, and etc.
    The last step couldn’t be done due to the file taking up too much memory.

The segments I am trying to separate only have slight density differences. So I tried to work in high resolution to not lose any data. Maybe I was too greedy?

I saw somewhere that the BMP files create vector volumes and should be converted to scalar volume.
I tried to do so, but my volumes were already scalar.
I think I am doing something terribly wrong with my ignorance, but I just can’t figure out what.
I wish I could attach my BMP files, but I don’t have my laptop with me at this moment.

I took a quick look at your scene file. I cannot load it to my laptop with 32GB of RAM either. The contents of the mrb files shows quite a few segmentation and a couple large volumes. I can load some of those individually without a problem, but I think the allocation problem happens when you try to load all of them.

Slicer has to keep everything loaded into the scene in the memory. So may not have sufficeint memory to load all of them at once. You can try to increase the virtual memory to get rid off the allocation error. Or selectively load a few things. MRB is just a zip file, you can decompress it like a zip archive and load the individual elements from it.

So for your last step, you can just load one volume, one segment to calculate area. Although you could have done this without separating your structures to 8 different subvolumes and loading them back individually into the same scene (which essentially doubled your memory consumption).

Thank you so much for your reply.
It is great to know that it is not the computer or the program’s fault!
I will try to save the volumes individually.

May I ask another question?
The reason I divided each volume was because the sectioning position is really important for me.
I have not done it in this file but I tried to reposition each samples to achieve sectioning images at the same position.
Can this be done without separating volumes? any suggestions?

Also, I wanted to get segment cross-section area of the transformed volume but couldn’t find a way to do so.
Is there a way to get segment cross section area of the transformed volume?

Thank you always for help.

First, I don’t think segment cross-sectional area works with transformed volumes at this point. There was some discussion but I am not sure if anythinng has changed. @lassoan ?

As for your memory issue, if your goal is to segment individual elements so that you can reorient them in an arbitrary plane, you should really use the ImageStack’s tools ROI option. That way you can reduce your memory usage considerably, since you will be importing only one specimen specimen at a time. Yes, this means you have to run ImageStacks eight times (since you have eight specimens in that scan), but that’s still considerably faster and memory efficient than using the segmentations.

1 Like

Thank you for your advice
As you said, it probably is much faster to work with smaller 8 files than waiting for my monster to work.
I will do accordingly.

For the segment cross-sectional area, I have read the discussion you have mentioned.
I am assuming the module status is the same since then.
I will just resample volume to work around the problem until any change is made.

One last question though, it seems like the original volume is needed as a reference volume to avoid the product being cropped during the resampling process.

Is there any other way to prevent this without the original volume?
So I can delete the original volume for the process optimization?

As far as I know the original volume is only used as a reference geometry to set the geometry of the newly resampled volume. If your rotations moving the target out of the reference volume, then it will get cropped. In that case you need to make your reference volume in larger dimensions. But other may chime in.

We (mostly @chir.set) worked a lot on cross-sectional measurements recently. I think the latest version of Centerline metrics module provides cross-sectional area measurements along arbitrary axis (or curve) without the need to transform the volume.

Unfortunately, ‘Centerline metrics’ requires a markups curve that have ‘Radius’ scalars, i.e, produced by ‘Extract centerline’. It can of course be updated to work on arbitrary markups curve, except that the UI would become confusing.

I think there should be a separate module specially designed to calculate surface areas of models/segments using arbitrary markups curve. Or add such a functionality to ‘Segment cross-section area’ module, based on ‘Centerline metrics’ approach.

@hherhold

We’ll remove that limitation. We cannot maintain 3 modules, each with an arbitrary set of limitations. We can easily make it one module for exploring and quantifying centerlines.