Presents the 'Brazilian Fauna Virtual Anatomic Collection'

[Rafael_V_Monteiro]

Operating system: Ubuntu 22.04.3 LTS (GNU/Linux 6.5.0-21-generic x86_64)
Slicer version: 5.6.0

Greetings Slicer community!

I am here to share the success of using Slicer3D to build our Brazilian Fauna Virtual Anatomic Collection, from University Federal of Juiz de Fora (UFJF) - MG, Brazil, hosted at:

CAVFB institutional page (in Brazilian Portuguese).

Collection access (English/Brazilian Portuguese)

There are many Brazilian wild animls species threatened by agricultural production, deforestation, chemical pollution, diseases and human intervention. However, such huge species diversity lacks proper documentation in the scientific literature, as many species are unknown in their simplest aspects, as anatomic bone features. In many cases, skeletons and bones are available in Biological Collections, maintained by numerous institutions, in Brazil or abroad. To consult or handle these collections’ valuable biological samples in person, visits are required, as well the authorization from the collections’ curators. Both requirements might be difficult to fulfill, as long expensive travels and/or absence of a considered proper purpose may hinder such access.

Our virtual collection was designed to meet these requirements for anyone who wants to know, study or simply hold bones and skeletons of Brazilian fauna: we exibhit these osteological structures online as 3D models, which are created from wild animals carcasses officially handed over to us by Brazilian governamental environment agencies. Such 3D models are freely downloadable as files fit to be printed.

All this would be unatainable if wasn’t by Slicer3D and other companion apps. Our collection project started as an idea without tools. In the course of the last two and a half years I had become acquainted with Slicer3D software. Although I knew about it, internet research has made clear for me that it was the right tool to the task ahead: 3D bone modelling and segmentation. After this, the challenge was how to properly show 3D models online in a pleasing and scientific way.

It was all sorted out when I learned about OpenAnatomyExport module and Online3Dviewer website, which I am adapting to our purposes. These two tools were a game changer for displaying our models. I also adapted scripts from Slicer script repository for generating models of the topographic features of the bones. Such acomplishments would not be achieved without Slicer3D community, whose help is most thanked.

Today, our research team includes me, four professors from University Federal de Juiz de Fora, and three graduate students. Past those almost three years of project, five other students had participated on this research project. We are presenting a paper in a national engineering congress in Brazil, and expect produce and help others to produce good scientific data about Brazilian wild fauna.

Best regards,

Prof. PhD. DVM Rafael V Monteiro

CAVFB Curator

[muratmaga]

Thanks for sharing this. Looks like you guys have done a great start, looking forward to seeing the increased diversity. I do have couple comments/suggestions.

1. If you can please register your server, and provide a fully qualified domain name. IP address only servers may cause issue with firewalls and proxies in unexpected ways.
2. It would be great if you can provide some specimen collection information, as well as imaging modality used and the underlying data (since these all appear as segmented models).
3. Some of the samples are fairly heavily decimated, which I believe is done to increase the interactivity of the 3D scene. But unfortunately it caused holes and rips (eg. scapula and ischium in the marmoset). You may want to selectively decimate the models.

While these interactive models are useful as teaching aids/visuals, and can possibly used for 3D printing, the label data (segmentation), is probably far more high resolution than these. We are starting a new project to make exchanging such label data more easily. Would you be interested in contributing your segmentation to that?

Again, great work. Glad that Slicer and other open-source tools have been useful for your project.

[tsehrhardt (Terrie Simmons)]

Love this! One thing you can do is add hierarchies so users can toggle the visibility of groups of bones as well–this works well with 3Dviewer. I have done this by adding “empties” to the glb/gltf in Blender and parenting each empty to the relevant bones. You can even have nested empties as shown in my attached image.

Happy to help you out with that if you’re interested.

[lassoan (Andras Lasso)]

Indeed, hierarchy is very useful! OpenAnatomy export module preserves the subject hierarchy in the exported model, so you can create the hierarchy in Slicer (right-click to create folders and drag-and-drop models into them in the Data module) and export glTF with hierarchy by a single click.

[rkikinis]

https://ta2viewer.openanatomy.org/?id=3932
This is an example using TA2 human anatomy

[tsehrhardt (Terrie Simmons)]

That’s good to know. Thanks!

[Rafael_V_Monteiro]

Thank you, sir, for your kind message, compliments and comments.
Regarding your suggestions:

@muratmaga

If you can please register your server, and provide a fully qualified domain name. IP address only servers may cause issue with firewalls and proxies in unexpected ways.

I am aware of those issues you mentioned, and I am ensuring that the University IT department registers the domain, as this IP number is a governamental one.

@muratmaga

It would be great if you can provide some specimen collection information, as well as imaging modality used and the underlying data (since these all appear as segmented models).

I am gradually adapting the 3Dviewer template to display such data, as well statistics about the segments. I am confident that I will achieve this soon .

@muratmaga

Some of the samples are fairly heavily decimated, which I believe is done to increase the interactivity of the 3D scene. But unfortunately it caused holes and rips (eg. scapula and ischium in the marmoset). You may want to selectively decimate the models.

We define a minimum Hounsfield Unit (HU) for each specimen’s DICOM images to be segmented. This minimum HU range around 150-200 HU units, above which everything will be considered bone. The holes and rips are due to presence of cancellous bone in those regions. If we lower the HU limit, many non-bone structures will bem included in the ‘skeleton’, such as tendons, salivary glands, stomach content and so on. Thus, we decided to allow these ‘failures’ as a demonstration of how calcification may vary among the species.

@muratmaga

Would you be interested in contributing your segmentation to that?

Absolutely! You may message me in private, perhaps?

@muratmaga

Glad that Slicer and other open-source tools have been useful for your project.

As a researcher from a developing country I strongly believe in FOSS, which use allow me to try to level the game. This project is a result of this.

Best regards!

[Rafael_V_Monteiro]

Than you for your message, compliments and suggestions.
I would like to add hierarchy to the segmentations, but I could not do that.
My unsuccesful steps:

1. In the data module, right-click over the segmentation name, select ‘Add child folder’.
2. However, the added folder does not allow any segment to be dragged in, as shown bellow (blue folder).

What am I doing wrong?

Again, thank you, best regards,

[Rafael_V_Monteiro]

Thank you for your suggestion. I tried it, but had no success. Please see previous reply, the added child folder does not allow any segment to be dragged in it.

What am I doing wrong?
Best regards,

[Rafael_V_Monteiro]

Thank you, I´ll try to accomplish that.
Best regards,

[pieper (Steve Pieper (Isomics, Inc.))]

Thank you @Rafael_V_Monteiro for sharing this valuable project

@Rafael_V_Monteiro

The holes and rips are due to presence of cancellous bone in those regions.

Yes, this is a very interesting aspect of the data and worth visualizing. I think the new ColorizeVolume method would be good with this data and you could use it to make movies to accompany the 3D surface models. If we like it, it should also be possible at some point to port this method to run natively in the browser.

Volume rendering colorized with segmentation

Yes, exactly. This image uses the CT as alpha, so you get nice detail with the coloring, but we should also do the option where the segmentation opacity controls the alpha channel. We’ll need to do the local smoothing / surface fitting in the GPU using the segmentation and the color from the lookup table. This should be effectively the same as building a surface model but faster and with some extra options. I agree we should think about how we should handle volume rendering from a UI perspe…

VTK multivolume/cinematic volume rendering

OMG! This is a game changer for us. We just now a way to adjust TF per segment basis somehow… And regular shadows/lights work great, just need the ambient shadows as @lassoan mentioned. [image]

[lassoan (Andras Lasso)]

@Rafael_V_Monteiro

Thank you for your suggestion. I tried it, but had no success. Please see previous reply, the added child folder does not allow any segment to be dragged in it.

Unfortunately, you cannot create folders within a segmentation node. Instead you can export the segmentation to models (by two clicks in Data module, using the right-click menu) and you can then drag-and-drop those models.

@Rafael_V_Monteiro

The holes and rips are due to presence of cancellous bone in those regions

I agree with @pieper that there are much, much better ways to indicate bone density than corrupting the segmentation with artifacts. The good news is that if you still have the segmentations then you may be able to fix some of these segmentation errors automatically using Wrap Solidify effect.

You can decide later if you want to show bone density and how (changing surface mesh transparncy, colors, etc., or use colored volume rendering when visualizing the scene in capable software, such as Slicer). You can show surface models in the browser by default but you could add an “Open in 3D Slicer” button that opens the 3D model directly in Slicer on your computer (using slicer:// custom URL protocol, as implemented during the last project week, or by simply adding a download link to a .mrb file).

[muratmaga]

@lassoan

se segmentation errors automatically using Wrap Solidify effect.

@Rafael_V_Monteiro
Here is an example how Colorize Volumes module can be used in junction with segmentation to give better rendering. You can find the Colorize Volume in Sandbox extension:

Unlike its segmentation, you can control how much of the bone resorption in the mandible to be visible in the volume rendering.

vs

[Rafael_V_Monteiro]

Hi Steve, thank you for your great remarks.
I will experiment with ColorizeVolume to see if I reach a so nice model you posted in the link. I agree that alpha values (i.e. opacity) should be used to express bone density, if I understood correctly the proposal.
I believe that is related to what causes the 3Dviewer measure tool to not work properly when I use transparency in the segments, with the objective of allowing bones’ canals and sinuses to be seen. The measure tool set the markers behind the structures and not in front.
I have much to learn of Slicer, and your comments and links will help me to go in the right direction, thank you.

Best regards,

[Rafael_V_Monteiro]

Hi Andras, I will follow your hints. Thank you for the help to put me in the right direction.
Best regards,

[Rafael_V_Monteiro]

Great hint Murat! I will explore that. Let´s keep in touch.
Best,

[lassoan (Andras Lasso)]

@Rafael_V_Monteiro

I believe that is related to what causes the 3Dviewer measure tool to not work properly when I use transparency in the segments, with the objective of allowing bones’ canals and sinuses to be seen

It is not well defined what should be the behavior when you click on a transparent surface - whether you want to pick the 3D position on the first surface (regardless of transparency), the first surface above a certain transparency threshold, the first surface where a certain cumulative opacity is reached, the first opaque surface, etc.

If you want to do measurements in a surface mesh based software then the best is not to use transparency (just clip or hide objects that occlude the view).

If you use 3D Slicer then you can do the same - measure on surfaces, or you can use colorized volume rendering. In volume rendering, opacity is continuously accumulated along each viewing ray - not just object boundaries - and the picked 3D position us where this opacity reaches 50%, which usually picks the 3D position you would expect.

[Rafael_V_Monteiro]

Got the problems. I have to think if it is functional to reopen a no-transparency model when user click on measure tool.
But I certainly will have a try in the ColorizeModel to address these questions.
Thank you very much, best regards,
Rafa