Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images

Abstract

Automatic and accurate detection of anatomical landmarks is an essential operation in medical image analysis with a multitude of applications. Recent deep learning methods have improved results by directly encoding the appearance of the captured anatomy with the likelihood maps (i.e., heatmaps). However, most current solutions overlook another essence of heatmap regression, the objective metric for regressing target heatmaps and rely on hand-crafted heuristics to set the target precision, thus being usually cumbersome and task-specific. In this paper, we propose a novel learning-to-learn framework for landmark detection to optimize the neural network and the target precision simultaneously. The pivot of this work is to leverage the reinforcement learning (RL) framework to search objective metrics for regressing multiple heatmaps dynamically during the training process, thus avoiding setting problem-specific target precision. We also introduce an early-stop strategy for active termination of the RL agent's interaction that adapts the optimal precision for separate targets considering exploration-exploitation tradeoffs. This approach shows better stability in training and improved localization accuracy in inference. Extensive experimental results on two different applications of landmark localization: 1) our in-house prenatal ultrasound (US) dataset and 2) the publicly available dataset of cephalometric X-Ray landmark detection, demonstrate the effectiveness of our proposed method. Our proposed framework is general and shows the potential to improve the efficiency of anatomical landmark detection.

Application of neural networks in prenatal diagnosis of atrioventricular septal defect

“…To date, artificial intelligence has been widely used in the FE area, and it is expected to improve the congenital cardiac abnormality detection rate by combining with traditional methods ( 10 , 11 ). In 2023, Veronese ( 26 ) used the fetal intelligent navigation echocardiography technology to realize the identification of AVSD, but it requires experienced sonographers to identify the views and the anatomical structures.…”

“…“Landmarks” are defined as some points or curves with specific characteristics and corresponding relationships in location and topology in the sense of medical anatomy ( 9 ). Automatic detection and assessment of landmarks, one of the Convolutional Neural Network algorithms, is an important and active topic in the field of medical image processing research ( 10 ). In the echocardiographic field, landmark detection was used to estimate the ventricular function ( 11 ), detect the prolapse location of the mitral valve ( 12 ), and detect the images’ border automatically ( 13 ).…”

Automatic Myotendinous Junction Identification in Ultrasound Images Based on Junction-Based Template Measurements

“…Deep neural networks (DNN) have significantly advanced various medical image analysis tasks, including anatomical landmark localization [25], [26], [27]. MTJ detection has also seen great promise in recent years with the availability of deeplearning models [28], [29], [30], [31].…”

“…Some DNN methods regress the MTJ positions by learning the map from the input appearance to the image coordinates [30], [31]. However, these methods require the network to learn a complicated mapping between a landmark's appearance and the corresponding image coordinates, making them usually challenging to train [27], [32]. Zhou et al [29] proposed a proposal-based DNN method to integrate an MTJ landmark regression branch into the Faster Region-based Convolutional Neural Network (Faster R-CNN) [33] architecture for adaptively segmenting the MTJ region more accurately.…”

A novel artificial intelligence model for fetal facial profile marker measurement during the first trimester

“…DL has proven to be an e cient tool for medical image-processing tasks by automatically extracting semantic features from images, 7,8 applications in prenatal ultrasound include object detection, 9,10 semantic segmentation, [11][12][13] and landmark localization. 14 Sun et al 15 proposed the Least Absolute Shrinkage and Selection Operator (LASSO) method, which incorporates fetal nuchal translucency (NT) thickness, along with various facial pro le markers, including pre-nasal thickness (PT) and MNM angle. It can serve as an e cient prognostic method for trisomy 21 during the rst trimester.…”