Automatic landmark correspondence detection in medical images with an application to deformable image registration

Abstract

Purpose: Deformable image registration (DIR) can benefit from additional guidance using corresponding landmarks in the images. However, the benefits thereof are largely understudied, especially due to the lack of automatic landmark detection methods for three-dimensional (3D) medical images.Approach: We present a deep convolutional neural network (DCNN), called DCNN-Match, that learns to predict landmark correspondences in 3D images in a self-supervised manner. We trained DCNN-Match on pairs of computed tomography (CT) scans containing simulated deformations. We explored five variants of DCNN-Match that use different loss functions and assessed their effect on the spatial density of predicted landmarks and the associated matching errors. We also tested DCNN-Match variants in combination with the open-source registration software Elastix to assess the impact of predicted landmarks in providing additional guidance to DIR.Results: We tested our approach on lower abdominal CT scans from cervical cancer patients: 121 pairs containing simulated deformations and 11 pairs demonstrating clinical deformations. The results showed significant improvement in DIR performance when landmark correspondences predicted by DCNN-Match were used in the case of simulated (p ¼ 0e 0 ) as well as clinical deformations (p ¼ 0.030). We also observed that the spatial density of the automatic landmarks with respect to the underlying deformation affect the extent of improvement in DIR. Finally, DCNN-Match was found to generalize to magnetic resonance imaging scans without requiring retraining, indicating easy applicability to other datasets.Conclusions: DCNN-match learns to predict landmark correspondences in 3D medical images in a self-supervised manner, which can improve DIR performance.

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review

“…Only one study utilized reinforcement learning [78], a technique where an agent learns to make decisions based on rewards and punishments. Regardless of the ML technique employed, training dataset sizes were generally small, with the three largest patient cohorts corresponding to auto-contouring studies (520-1108 patients) [49,51,86]. ML models often struggle with small sample sizes, especially when considering complex, multidimensional data like medical images, where models must learn intricate generalizable spatial relationships.…”

Anatomical Landmark Detection for Initializing US and MR Image Registration

“…Heinrich et al [9] proposed a landmark detection method specifically designed for lung computed tomography (CT) registration, which is not generalizable to other tasks. Grewal et al [10] presented DCNN-Match, that learns to predict landmark correspondences in lower abdominal CT scans and in a self-supervised manner, which significantly improves the performance in deformable image registration. Song et al [11] proposed an affine registration method for US and MR images based on four anatomical landmarks, which requires not only landmark detection network, but also segmentation network.…”

Bionic Artificial Neural Networks in Medical Image Analysis

“…Preprocessing improves the quality and relevance of the images by removing noise, enhancing contrast [ 12 ], and segmenting the regions of interest. Registration [ 13 ] aligns and merges multiple images of the same patient or anatomical region to enable accurate comparisons and analyses. Feature extraction [ 14 ] identifies and quantifies relevant characteristics or patterns in the image data to aid in diagnosis or treatment planning.…”