Heatmap-based 2D Landmark Detection with a Varying Number of Landmarks

Stern, Antonia; Sharan, Lalith; Romano, Gabriele; Koehler, Sven; Kretzler, Matthias; Simone, Raffaele De; Wolf, Ivo; Engelhardt, Sandy

doi:10.1007/978-3-658-33198-6_7

Cited by 10 publications

(33 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In earlier works on a small intra-operative dataset [8,9], we have used random forests and tailored post-processing for point detection and optical flow for point tracking. Our previous work on the same data base [24] formulates the landmark detection task as a deep learning-based approach, and demonstrates first results on intra-operative and surgical simulator datasets for heart surgeries. Hervella et al [15] demonstrated a similar method for the case of retinal fundus images.…”

Section: Heatmap-based Approachesmentioning

confidence: 88%

“…Brosch et al [1] tackles this problem by using a novel objective function. In this paper, we extend our previous work [24] and tackle the unbalanced multi-instance sparse-segmentation task through the use of a differentiable convolutional Soft-Argmax layer combined with a balanced loss function. Iqbal et al [16] used a differentiable Soft-Argmax layer to extract the landmark locations from the heatmap, but the problem formulation contained a single heatmap per channel for a pre-defined number of heatmaps.…”

Section: Heatmap-based Approachesmentioning

confidence: 96%

“…The approach is based on a heatmap which typically models the distribution of likelihood around the point as a Gaussian. In this paper, we extend on our previous conference work [24], where we proposed a simple U-net with a single channel output. The output map was thresholded and the centre of mass was calculated for each region to determine the final position of points.…”

Section: Introductionmentioning

confidence: 93%

“…Additionally, an F β loss is used to take into account the precision and recall for optimisation. Results compared to our previous baseline [24], along with ablations with various loss functions and output layer configurations, are presented.…”

Section: Heatmap-based Approachesmentioning

confidence: 99%

“…The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/. To be consistent with the challenge, we used the same data set split like in the challenge, which is slightly different from [24].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

et al. 2021

Self Cite

View full text Add to dashboard Cite

Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. Results: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean $$F_1$$ F 1 of $$+ 0.0422$$ + 0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean $$F_1$$ F 1 of $$+ 0.0865$$ + 0.0865 over the baseline. Conclusion: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.

show abstract

Section: Heatmap-based Approachesmentioning

confidence: 88%

Section: Heatmap-based Approachesmentioning

confidence: 96%

Section: Introductionmentioning

confidence: 93%

Section: Heatmap-based Approachesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Improved Heatmap-Based Landmark Detection

Yao¹,

Zhang²,

Li³

2021

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Heatmap Regression for Lesion Detection Using Pointwise Annotations

Myers-Colet¹,

Schroeter²,

Arnold³

2022

Medical Image Learning With Limited and Noisy Data

View full text Add to dashboard Cite

In many clinical contexts, detecting all lesions is imperative for evaluating disease activity. Standard approaches pose lesion detection as a segmentation problem despite the time-consuming nature of acquiring segmentation labels. In this paper, we present a lesion detection method which relies only on point labels. Our model, which is trained via heatmap regression, can detect a variable number of lesions in a probabilistic manner. In fact, our proposed post-processing method offers a reliable way of directly estimating the lesion existence uncertainty. Experimental results on Gad lesion detection show our point-based method performs competitively compared to training on expensive segmentation labels. Finally, our detection model provides a suitable pre-training for segmentation. When fine-tuning on only 17 segmentation samples, we achieve comparable performance to training with the full dataset.

show abstract

Heatmap-based 2D Landmark Detection with a Varying Number of Landmarks

Cited by 10 publications

References 7 publications

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Improved Heatmap-Based Landmark Detection

Heatmap Regression for Lesion Detection Using Pointwise Annotations

Contact Info

Product

Resources

About