Is dataStore upload working?

Support
1

I am trying to upload a scene to DataStore. It has a two volumes (one grayscale, one labelmap) and a markups list. I created an account, logged in, described the contents etc. Now the upload is hung, doesn’t seem to progress et all. When saved to disk, MRB file itself is only 11MB. Is that a connection issue (I am behind a proxy)?

2

Are you trying to use the upload feature in Slicer or just the Midas web interface in a web browser?

Midas is being replaced by Girder and maybe some features have been disabled already.

3

@lassoan
No, I am using the built-in upload option. See the screenshot.
image

4

You can try uploading scenes by using the web interface in a browser. What data would you like to upload and for what audience and purpose?

When the DataStore module was developed it was hard to find reliable and easily affordable hosting service for research data. However, there are lots of possibilities now (GitHub, Zenodo, low-cost unlimited hosting services, etc.), so DataStore may be reduced to a data index that can point to data stored at various places - similarly how the Extension Manager or Sample Data module work.

You can probably find good free hosting service and you can add links and images to SampleData (as it is done for example in SequenceSampleData module in Sequences extension).

5

I don’t need hosting, just a to convenient way to pull existing data into Slicer. This looks it is going to work a like champ. Thanks Andras.

6

Actually one more question: I modified the example script with my url. Data downloads correctly, but fails to get loaded into Slicer. I can open the cached version manually, it is not corrupted or anything. All I changed are these lines:

sampleName=‘Mouse Head microCT Atlas’,
uris=‘https://github.com/muratmaga/mouse_CT_atlas/blob/3da1c36c057537384376155f347d08b34817fa5c/data/templates/35_mic_23strain_mus_template_UCHAR.nii.gz?raw=true’,
fileNames=‘35_mic_23strain_mus_template_UCHAR.nii.gz’,
nodeNames=‘Mus’,

<b>Requesting download</b> <i>35_mic_23strain_mus_template_UCHAR.nii.gz</i> from https://github.com/muratmaga/mouse_CT_atlas/blob/3da1c36c057537384376155f347d08b34817fa5c/data/templates/35_mic_23strain_mus_template_UCHAR.nii.gz?raw=true…

<i>Downloaded 960.0 KB (10% of 9.3 MB)…</i>

<i>Downloaded 1.9 MB (20% of 9.3 MB)…</i>

<i>Downloaded 2.8 MB (30% of 9.3 MB)…</i>

<i>Downloaded 3.7 MB (40% of 9.3 MB)…</i>

<i>Downloaded 4.7 MB (50% of 9.3 MB)…</i>

<i>Downloaded 5.6 MB (60% of 9.3 MB)…</i>

<i>Downloaded 6.5 MB (70% of 9.3 MB)…</i>

<i>Downloaded 7.5 MB (80% of 9.3 MB)…</i>

<i>Downloaded 8.4 MB (90% of 9.3 MB)…</i>

<i>Downloaded 9.3 MB (100% of 9.3 MB)…</i>

<b>Download finished</b>

<b>Requesting load</b> <i>35micron_Mouse_Atlas</i> from C:/Users/Murat/AppData/Local/Temp/Slicer/RemoteIO/35_mic_23strain_mus_template_UCHAR.nii.gz…

<font color="red"><b> Load failed!</b></font>

7

@muratmaga here’s a working sample for that data.

import SampleData

dataSource = SampleData.SampleDataSource(
  sampleName= 'Mouse Head microCT Atlas', 
  uris= 'https://github.com/muratmaga/mouse_CT_atlas/blob/3da1c36c057537384376155f347d08b34817fa5c/data/templates/35_mic_23strain_mus_template_UCHAR.nii.gz?raw=true',
  fileNames= '35_mic_23strain_mus_template_UCHAR.nii.gz', 
  nodeNames= 'Mus'
  )

SampleData.SampleDataLogic().downloadFromSource(dataSource)
8

Thanks Steve, it works if I copy/paste to the python console.

But I actually want to add a new Category to the SampleData module as the Sequences module have done. I have been trying to modify the following to make it work with this particular data (as well as few other different kinds).

9

This works for me:

    SampleData.SampleDataLogic.registerCustomSampleDataSource(
      sampleName='Mouse Head microCT Atlas',
      uris='https://github.com/muratmaga/mouse_CT_atlas/blob/3da1c36c057537384376155f347d08b34817fa5c/data/templates/35_mic_23strain_mus_template_UCHAR.nii.gz?raw=true',
      fileNames='35_mic_23strain_mus_template_UCHAR.nii.gz',
      nodeNames='Mus',
      )
10

This works for me too, thanks @lassoan.

11

Works for me too as it is.
So, loadFileType=‘Volume Sequence’ tag is not necessary, then?

12

You need to define “Volume Sequence” if you want the data set to be loaded as a volume sequence (4D volume).

13

Is there some documentation for the tags that I have to use and what they can be set as ? I am trying with a mrb scene now, and I had the same issue as before, ie., downloads but doesn’t load into Slicer.

14

You can use standard Python help method to get documentation of any method or class.

>>> import SampleData
>>> help(SampleData.SampleDataLogic.registerCustomSampleDataSource)

Help on function registerCustomSampleDataSource in module SampleData:

registerCustomSampleDataSource(category='Custom', sampleName=None, uris=None, fileNames=None, nodeNames=None, customDownloader=None, thumbnailFileName=None, loadFileType='VolumeFile', loadFileProperties={})
    Adds custom data sets to SampleData.
    :param category: Section title of data set in SampleData module GUI.
    :param sampleName: Displayed name of data set in SampleData module GUI.
    :param thumbnailFileName: Displayed thumbnail of data set in SampleData module GUI,
    :param uris: Download URL(s).
    :param fileNames: File name(s) that will be loaded.
    :param nodeNames: Node name in the scene.
    :param customDownloader: Custom function for downloading.
    :param loadFileType: file format name ('VolumeFile' by default).
    :param loadFileProperties: custom properties passed to the IO plugin.

File reader plugin name (loadFileType) is defined in each reader plugin. New plugins can be specified in any module in any extension, so there is no “list”. You can find most of them mentioned in https://github.com/Slicer/Slicer/blob/master/Base/Python/slicer/util.py - search for filetype.

15

Thanks Andras. This is very useful.

1 Like
16

The issue with loading an MRB file when registering a custom data source is documented here and fixed in r27666. Thanks for your advice @lassoan.