# Train AI for landmark detection

**URL:** <https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734>\
**Category:** Support\
**Created:** [April 14, 2026, 6:29pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734 "2026-04-14T18:29:17Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mau\_igna\_06](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/mau_igna_06/32/9056_2.png) [@mau\_igna\_06](https://discourse.slicer.org/u/mau_igna_06)\
**Post date:** [April 14, 2026, 6:29pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/1 "2026-04-14T18:29:17Z")

</div>

Hi,

I’d like to ask here which are community suggestions about selecting an AI model and setting up training data for a medical landmark detection task.

In my use case, I need to detect the superior tip of healthy kidneys on CBCTs. I have a training dataset with around 150 CBCTs with the corresponding landmarks (CBCT on .nrrd files and points in .mrk.json files). I think my training data (i.e. sup kidney tips) could have till a 1cm error. Current training shows 70% accuracy and I don’t expect it to get higher. So I assume the model won’t work after training but that will have to wait a few days (till the training ends)

More information: same use case with same AI model but targeting CTs instead of CBCTs for training, even with halve of training data (e.g. around 70 CTs), had 95% accuracy during training and my tests show it working

---

<div class="post-metadata">

**Author:** ![IVarha](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/ivarha/32/66704_2.png) [@IVarha](https://discourse.slicer.org/u/IVarha)\
**Post date:** [April 15, 2026, 8:12am UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/2 "2026-04-15T08:12:36Z")

</div>

Hello,

could you please elaborate what is in your case accuracy? I am curious because from reading your description you have to use distance metrics instead of accuracy.

---

<div class="post-metadata">

**Author:** ![JBeninca](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/jbeninca/32/10039_2.png) [@JBeninca](https://discourse.slicer.org/u/JBeninca)\
**Post date:** [April 16, 2026, 12:24pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/3 "2026-04-16T12:24:41Z")

</div>

if the points are paired, you can use the algorithm:

> **[Iterative Closest Point (ICP) for 3D Explained with Code](https://learnopencv-com.translate.goog/iterative-closest-point-icp-explained/?_x_tr_sl=en&_x_tr_tl=es&_x_tr_hl=es&_x_tr_pto=tc)**
>
> Iterative Closest Point (ICP) explained with code in Python and Open3D which is a widely used classical algorithm for 2D or 3D point cloud registration

> **[Iterative Closest Point Algorithm - an overview | ScienceDirect Topics](https://www-sciencedirect-com.translate.goog/topics/engineering/iterative-closest-point-algorithm?_x_tr_sl=en&_x_tr_tl=es&_x_tr_hl=es&_x_tr_pto=tc)**

---

<div class="post-metadata">

**Author:** ![lassoan](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/lassoan/32/13_2.png) [@lassoan](https://discourse.slicer.org/u/lassoan)\
**Post date:** [April 17, 2026, 3:27pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/4 "2026-04-17T15:27:21Z")

</div>

Kidney detection on CBCT is a hard problem, while on CT it is an easy problem. First of all, the field of view of a CBCT is much smaller (just 20-30cm, so you don’t always see the whole kidney and surroundings), CBCT soft tissue contrast is much worse (mainly due to physics - scattering of the cone beam), and CBCT images are less standardized (voxel values may not correspond accurately to HU). So, the 70% vs 95% accuracy is expected. You may need much more data to get much better results.

What model do you use now?

You can try [nnLandmark](https://github.com/MIC-DKFZ/nnLandmark), an nnU-Net based landmark detection model. Unfortunately, it is not packaged cleanly (they forked and modified nnU-Net), but it should worth a try.

---

<div class="post-metadata">

**Author:** ![mau\_igna\_06](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/mau_igna_06/32/9056_2.png) [@mau\_igna\_06](https://discourse.slicer.org/u/mau_igna_06)\
**Post date:** [April 17, 2026, 5:41pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/5 "2026-04-17T17:41:32Z")

</div>

> [@lassoan](#):
>
> Kidney detection on CBCT is a hard problem, while on CT it is an easy problem.

Yes, I realized this while exploring my datasets and the early results I had.

> [@lassoan](#):
>
> You may need much more data to get much better results.

Yes, I expect the training loss to be reduced logarithmically by doubling the training data by the experience I have.

> [@lassoan](#):
>
> What model do you use now?

I’m using a RL model.

> [@lassoan](#):
>
> You can try [nnLandmark](https://github.com/MIC-DKFZ/nnLandmark), an nnU-Net based landmark detection model.

Yes, I have been researching other models (as well as hyperparameters optimization for the current model I’m using) for this task and I did find nnLandmark suggested.

Thanks a lot for your feedback

---

<div class="post-metadata">

**Author:** ![muratmaga](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/muratmaga/32/3622_2.png) [@muratmaga](https://discourse.slicer.org/u/muratmaga)\
**Post date:** [April 17, 2026, 5:53pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/6 "2026-04-17T17:53:52Z")

</div>

Are your CBCT’s are as standardized as your CTs?

In my experiments, I found the training for landmark detection being quite sensitive to positional differences. You may want to check whether CBCTs are more variable compared to CT, and if they are try normalizing the difference by registering to a standard orientation (and of course update your landmark coordinates accordingly) and then redo the training. Additionally some CBCTs i have seen are not normalized for intensities (ie., in 16 bit values exceeding HU values). You might also check whether your pipeline is doing standardization of intensities.

---

<div class="post-metadata">

**Author:** ![mau\_igna\_06](https://sea2.discourse-cdn.com/flex002/user_avatar/discourse.slicer.org/mau_igna_06/32/9056_2.png) [@mau\_igna\_06](https://discourse.slicer.org/u/mau_igna_06)\
**Post date:** [April 17, 2026, 6:16pm UTC](https://discourse.slicer.org/t/train-ai-for-landmark-detection/46734/7 "2026-04-17T18:16:18Z")

</div>

> [@muratmaga](#):
>
> Are your CBCT’s are as standardized as your CTs?  
> In my experiments, I found the training for landmark detection being quite sensitive to positional differences.

I’ve tried that. I have tried to be rigorous in the creation and review of the training data pairs (i.e. images and landmarks).

> [@muratmaga](#):
>
> You may want to check whether CBCTs are more variable compared to CT

I’ve found that around 5% of the original CBCTs in my dataset are out of distribution (i.e. outliers) and I have just skipped them during training as I don’t expect to be doing inference on such bad quality CBCTs.

> [@muratmaga](#):
>
> You might also check whether your pipeline is doing standardization of intensities.

Yes, as far as I remember that is being taken cared of