Deep learning model for segmenting many of same things

This is not really Slicer related, but maybe relevant in context of upcoming PW.

Is anyone aware of a DL architecture that can be used to train and segment things that look similar, but doesn’t necessarily have a fixed count. Biological examples would be vertebra in fishes or snakes, tooth in alligators, crocodyles, salamanders, scales in lizards and many other animals, cell nuclei etc…

The idea would be the user would only label a few of those those in a given dataset, and the model would extract all of them, regardless of how many there are.

I don’t think current implementations of UNet would work, since you have to designate all the classes beforehand.

1 Like

Have you tried the Segment Anything Models? They take. Seems like they would be well suited to this if you are able to provide some input about which structure you want.

It would be nice to have an AI that can segment all bones in a CT as separated structures on a single segment (this would need to ignore cartilage I guess). Then using islands effect you could separate them easily :slight_smile:

I tried sometime ago, but didn’t manage to get it working (neither as extension nor as GH).

Agreed, there’s more work to be done but SAM seems like a good approach for what you asked.