NVIDIA Clara AI-Assisted Annotation Extension

@lassoan I spoke with one if my colleagues here and standing up an annotation server for development is something we can help with. May take a couple of days but will keep you posted. Will post both here and on repository with details.

Exciting turn around on this work!

2 Likes

Thank you… I have been referring the same extensions to start with.
One thing I couldn’t find: when user clicks label (selects label like Spleen) I would like to send a request to AIAA server and keep the list of models ready… instead of additional click to “Fetch Models”

Is there a way I can get an event that label is selected from the labelmap?

If you mean that you would like to know when the user changes the segment name (or double-clicks the color swatch and selects a terminology item) then you can add an observer to the SegmentModified event in the segmentation node. Whenever this event is invoked your observer can retrieve the segment’s name or terminology and request the appropriate model.

@lassoan I think the question is how to get the event when the segment is selected in the segment list.

@SachidanandAlle in the QuantitativeReporting module we had a similar situation where we wanted to configure actions based on segment selection. I might not remember it correctly, but I think Christian (@che85) who implemented that module didn’t find a way to get that event, and because of that had to customize Segment Editor in this class. I would’t be able to list all the reasons why that customization was needed, beyond this event handler.

To get a notification immediately when the user selects a different segment, you can use the currentSegmentIDChanged signal of the segment editor widget. If you want to check anytime what is the currently selected segment, you can get it from the segment using currentSegmentID method (or from the segment editor MRML node).

1 Like

Just in case somebody comes across this topic now: NVidia Clara AIAA extension is now available in extension manager.

Hello,

Does anyone knows why this message appears ? Is there any server configuration to be made?

Thanks

Sorry, it seems that the latest updates to the extension broke the network communication. It should be fixed by tomorrow but until then you can use this release: https://download.slicer.org/?date=2020-02-20

1 Like

Has anyone tried using DeepGrow? @lassoan I recall we talked that it was 2D only, but their documentation implies it’s 3D (@erikziegler pointed this out).

https://docs.nvidia.com/clara/tlt-mi/clara-train-sdk-v3.0/aiaa/key_features.html#deepgrow

Yes, at the time it was released it was 2D, maybe it is 3D now. However, it does not use any underlying organ model, so it seems to be just another variant of Grow from seeds/Watershed/etc.

Nevertheless, it would be still interesting to try. That would require setting up the latest version of the AIAA server. @pieper if you have time to set it up then I can give you access to the server.

Hi There I am also getting the error.
“Failed to fetch models from remote server. Make sure server address is correct and <server_uri>/v1/models is accessible in browser”
Using a Windows machine and the url field is blank as advised.
Just downloaded the extension yesterday. How can I resolve?
Thanks a lot

The server is up and running and works well from Slicer, from a Windows machine (I’ve just tried it now). If you are behind a company or hospital firewall that prevents web requests to go through then you may need to specify the web access proxy as described here.

Are there any models for segmenting the Bronchi from the lungs?
Thanks a lot

I’ve updated the demo Slicer AI server to use latest NVidia Clara 3.0 version and uploaded the latest models (NVidia provides over 20 image segmentation models as technology demonstration - you can see the list here: https://ngc.nvidia.com/catalog/collections/nvidia:claratrainframework). You may find the fully automatic lung segmentation model useful.

In addition to this, you may find models at other places that can be loaded into the AIAA server directly, or brought into AIAA compatible format (see instructions for bringing your own model).

Hi Andras,

Do you know if there are any open-source COVID lung lesion models currently available in Clara, or elsewhere? And are you still providing a server to host models for the general public via AIAA Slicer plug-in?

Hope you are well.

-Brett

Yes, we still provide the server. We are in the process of switching to Clara 3.0, which has a few COVID related models, for example a fully automatic lung segmentation model. See complete list by clicking on “Download COVID-19 models” button on this page.

This is great, thank you!

Hello Andras,

I have just started to explore the possibilities of Clara and its extension to 3DSlicer. I am quite impressed with the concept and how it works together. My main focus of interest is liver segmentation. There are two liver segmentation models provided by Nvidia and also a general 2D Deepgrow method. Unfortunately, I could not find any annotation model for liver which would make the liver segmentation much faster compared to 2D Deepgrow. I saw the annotation models in some older Clara videos by Nvidia but actual NGC catalog does not offer them anymore. Is there any place where I could download them? Or maybe, do you have any recommendations how to create them from the existing segmentation models?

Thank you,

Petr

NVidia developed these models as technology demos, but has no intention of maintaining them. Instead, they provide funding to the monai community to create and maintain models that can be used in Clara. I would recommend to check out monai, see what models they have, and maybe ask on their discussion forum about what they recommend for liver segmentation.

A post was split to a new topic: NVIDIA AIAA server internal error