A review of deep learning-based three-dimensional medical image registration methods

Abstract

Medical image registration is a vital component of many medical procedures, such as imageguided radiotherapy (IGRT), as it allows for more accurate dose-delivery and better management of side effects. Recently, the successful implementation of deep learning (DL) in various fields has prompted many research groups to apply DL to three-dimensional (3D) medical image registration. Several of these efforts have led to promising results. This review summarized the progress made in DL-based 3D image registration over the past 5 years and identify existing challenges and potential avenues for further research.The collected studies were statistically analyzed based on the region of interest (ROI), image modality, supervision method, and registration evaluation metrics. The studies were classified into three categories: deep iterative registration, supervised registration, and unsupervised registration. The studies are thoroughly reviewed and their unique contributions are highlighted. A summary is presented following a review of each category of study, discussing its advantages, challenges, and trends. Finally, the common challenges for all categories are discussed, and potential future research topics are identified.

A dual‐supervised deformation estimation model (DDEM) for constructing ultra‐quality 4D‐MRI based on a commercial low‐quality 4D‐MRI for liver cancer radiation therapy

“…Preparing reference DVFs can be time-consuming and may limit the application of dual supervision. [24][25][26] However, the reference DVFs were only needed during model training, and dual supervision did not with a mean error <1 mm in all directions, suggesting that our method accurately reflects tumor motion. Also, the motion trajectories show no significant difference between UQ T1w and T2w 4D-MRI, which is expected since they were derived from the same DVF.…”

“…Two aspects of this approach are challenging: potential differences in the respiratory phases of the 3D MR images used as prior images and those of the 4D-MRI frames and the use of deep learning for image registration with several different types of image contrast. [24][25][26] Therefore, an automatic frame selection strategy was used instead of registering 3D MR images to every original 4D-MRI frame. Cross-correlation was computed between all 4D-MRI frames and the corresponding 3D MRI (T1w or T2w).…”

Advances in MRI‐guided precision radiotherapy

“…Recently, the rapid development of artificial intelligence (AI) is expected to perform image processing tasks such as superposition, registration, and segmentation more efficiently and precisely, which may widen the clinical application of 4D-MRI. 52 For example, MRF has great potential to improve MRI-guided radiotherapy workflow. 63 Contrary to conventional qualitative MRI scan schemes, the quantitative tissue maps derived from MRF demonstrated high repeatability and reproducibility.…”

“…The limitation of image processing speed impedes the clinical application of 4D‐MRI in a real‐time approach. Recently, the rapid development of artificial intelligence (AI) is expected to perform image processing tasks such as super‐position, registration, and segmentation more efficiently and precisely, which may widen the clinical application of 4D‐MRI 52 . For example, Figure 6 demonstrates synthetic ultra‐quality 4D‐MRIs from the original 4D‐MR image.…”

Deep learning-based fully automated differential diagnosis of eyelid basal cell and sebaceous carcinoma using whole slide images

“…The whole slide scanning technique enables H&E-stained sections to be digitized as whole slide images (WSIs), making it possible to develop computer-aided diagnostic methods. Due to the rapid development and wide application of deep learning (DL) in medical image field ( 22 , 23 ), there have been remarkable breakthroughs in computer-aided diagnostic methods based on WSI ( 24 - 27 ). Jain et al developed an Inception v3-based classification method to predict the tumour mutational burden using WSIs at different resolutions (×5, ×10, and ×20 magnification) ( 28 ).…”