Lung‐CRNet: A convolutional recurrent neural network for lung 4DCT image registration

Lu, Junlin; Jin, Renchao; Song, Enmin; Ma, Guangzhi; Wang, Manyang

doi:10.1002/mp.15324

Cited by 20 publications

(14 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The graphics processing unit (GPU) was NVIDIA GeForce GTX 1080Ti with 11 GB of graphics memory. Three deep learning-based registration frameworks were chosen as the base models: VoxelMorph (Balakrishnan et al 2019), ViT-V-Net (Chen et al 2021a), and CRNet (Lu et al 2021).…”

Section: Training and Testingmentioning

confidence: 99%

See 1 more Smart Citation

A discontinuity-preserving regularization for deep learning-based cardiac image registration

Lu¹,

Jin²,

Wang³

et al. 2023

Phys. Med. Biol.

Self Cite

View full text Add to dashboard Cite

Objective. Sliding motion may occur between organs in anatomical regions due to respiratory motion and heart beating. This issue is often neglected in previous studies, resulting in poor image registration performance. A new approach is proposed to handle discontinuity at the boundary and improve registration accuracy.Approach. The proposed discontinuity-preserving regularization (DPR) term can maintain local discontinuities. It leverages the segmentation mask to find organ boundaries and then relaxes the displacement field constraints in these boundary regions. A weakly supervised method using mask dissimilarity loss (MDL) is also proposed. It employs a simple formula to calculate the similarity between the fixed image mask and the deformed moving image mask. These two strategies are added to the loss function during network training to guide the model better to update parameters. Furthermore, during inference time, no segmentation mask information is needed.Main results. Adding the proposed DPR term increases the Dice coefficients by 0.005, 0.009, and 0.081 for three existing registration neural networks CRNet, VoxelMorph, and ViT-V-Net, respectively. It also shows significant improvements in other metrics, including Hausdorff Distance and Average Surface Distance. All quantitative indicator results with MDL have been slightly improved within 1%. After applying these two regularization terms, the generated displacement field is more reasonable at the boundary, and the deformed moving image is closer to the fixed image.Significance. This study demonstrates that the proposed regularization terms can effectively handle discontinuities at the boundaries of organs, improving the accuracy of deep learning-based cardiac image registration methods. Besides, they are generic to be extended to other networks.

show abstract

Section: Training and Testingmentioning

confidence: 99%

“…One is to perform pairwise registration independently for every two images in the image set, represented by VoxelMorph (Balakrishnan et al 2019) and ViT-V-Net (Chen et al 2021a), and the other is to simultaneously register multiple images. One example of the second category is CRNet(Lu et al 2021).…”

mentioning

confidence: 99%

A discontinuity-preserving regularization for deep learning-based cardiac image registration

Lu¹,

Jin²,

Wang³

et al. 2023

Phys. Med. Biol.

Self Cite

View full text Add to dashboard Cite

show abstract

“…CNNs significantly reduce the parameters of the hidden layer by sharing kernels. The encoder-decoder architecture is widely used in CNNs for medical imaging applications, such as image registration ( 27 - 29 ), image segmentation ( 30 - 33 ), and image synthesis ( 34 - 36 ).…”

Section: Introductionmentioning

confidence: 99%

Deep learning-based target tracking with X-ray images for radiotherapy: a narrative review

Liu,

Geng,

Huang

et al. 2024

Quant Imaging Med Surg

View full text Add to dashboard Cite

Background and Objective As one of the main treatment modalities, radiotherapy (RT) (also known as radiation therapy) plays an increasingly important role in the treatment of cancer. RT could benefit greatly from the accurate localization of the gross tumor volume and circumambient organs at risk (OARs). Modern linear accelerators (LINACs) are typically equipped with either gantry-mounted or room-mounted X-ray imaging systems, which provide possibilities for marker-less tracking with two-dimensional (2D) kV X-ray images. However, due to organ overlapping and poor soft tissue contrast, it is challenging to track the target directly and precisely with 2D kV X-ray images. With the flourishing development of deep learning in the field of image processing, it is possible to achieve real-time marker-less tracking of targets with 2D kV X-ray images in RT using advanced deep-learning frameworks. This article sought to review the current development of deep learning-based target tracking with 2D kV X-ray images and discuss the existing limitations and potential solutions. Finally, it also discusses some common challenges and potential future developments. Methods Manual searches of the Web of Science, and PubMed, and Google Scholar were carried out to retrieve English-language articles. The keywords used in the searches included “radiotherapy, radiation therapy, motion tracking, target tracking, motion estimation, motion monitoring, X-ray images, digitally reconstructed radiographs, deep learning, convolutional neural network, and deep neural network”. Only articles that met the predetermined eligibility criteria were included in the review. Ultimately, 23 articles published between March 2019 and December 2023 were included in the review. Key Content and Findings In this article, we narratively reviewed deep learning-based target tracking with 2D kV X-ray images in RT. The existing limitations, common challenges, possible solutions, and future directions of deep learning-based target tracking were also discussed. The use of deep learning-based methods has been shown to be feasible in marker-less target tracking and real-time motion management. However, it is still quite challenging to directly locate tumor and OARs in real-time with 2D kV X-ray images, and more technical and clinical efforts are needed. Conclusions Deep learning-based target tracking with 2D kV X-ray images is a promising method in motion management during RT. It has the potential to track the target in real time, recognize motion, reduce the extended margin, and better spare the normal tissue. However, it still has many issues that demand prompt attention, and further development before it can be put into clinical practice.

show abstract

“…17 Recently, there have been works published that use machine learning to predict radiation-induced pneumonitis, [18][19][20][21] derive ventilation [22][23][24][25][26] and perfusion maps [27][28][29][30] , and perform segmentation [31][32][33][34] and DIR. [35][36][37][38] However, no work to date has investigated using machine learning to model local pulmonary ventilation change following RT.…”

Section: Introductionmentioning

confidence: 99%

“…Currently, functional lung imaging and research are based on physical properties of the image such as calculating local CT density changes 16 or calculating regional volume changes using the Jacobian from deformable image registration (DIR) 17 . Recently, there have been works published that use machine learning to predict radiation‐induced pneumonitis, 18–21 derive ventilation 22–26 and perfusion maps 27–30 , and perform segmentation 31–34 and DIR 35–38 . However, no work to date has investigated using machine learning to model local pulmonary ventilation change following RT.…”

Section: Introductionmentioning

confidence: 99%

Predicting pulmonary ventilation damage after radiation therapy for nonsmall cell lung cancer using a ResNet generative adversarial network

et al. 2023

View full text Add to dashboard Cite

Background: Functional lung avoidance radiation therapy (RT) is a technique being investigated to preferentially avoid specific regions of the lung that are predicted to be more susceptible to radiation-induced damage. Reducing the dose delivered to high functioning regions may reduce the occurrence radiationinduced lung injuries (RILIs) and toxicities. However, in order to develop effective lung function-sparing plans, accurate predictions of post-RT ventilation change are needed to determine which regions of the lung should be spared. Purpose: To predict pulmonary ventilation change following RT for nonsmall cell lung cancer using machine learning. Methods: A conditional generative adversarial network (cGAN) was developed with data from 82 human subjects enrolled in a randomized clinical trial approved by the institution's IRB to predict post-RT pulmonary ventilation change. The inputs to the network were the pre-RT pulmonary ventilation map and radiation dose distribution. The loss function was a combination of the binary cross-entropy loss and an asymmetrical structural similarity index measure (aSSIM) function designed to increase penalization of underprediction of ventilation damage. Network performance was evaluated against a previously developed polynomial regression model using a paired sample t-test for comparison. Evaluation was performed using eight-fold crossvalidation. Results: From the eight-fold cross-validation, we found that relative to the polynomial model, the cGAN model significantly improved predicting regions of ventilation damage following radiotherapy based on true positive rate (TPR), 0.14±0.15 to 0.72±0.21, and Dice similarity coefficient (DSC), 0.19±0.16 to 0.46±0.14, but significantly declined in true negative rate, 0.97±0.05 to 0.62±0.21, and accuracy, 0.79±0.08 to 0.65±0.14. Additionally, the average true positive volume increased from 104±119 cc in the POLY model to 565±332 cc in the cGAN model, and the average false negative volume decreased from 654±361 cc in the POLY model to 193±163 cc in the cGAN model. Conclusions:The proposed cGAN model demonstrated significant improvement in TPR and DSC. The higher sensitivity of the cGAN model can improve the clinical utility of functional lung avoidance RT by identifying larger volumesThis is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

show abstract

Lung‐CRNet: A convolutional recurrent neural network for lung 4DCT image registration

Cited by 20 publications

References 42 publications

A discontinuity-preserving regularization for deep learning-based cardiac image registration

A discontinuity-preserving regularization for deep learning-based cardiac image registration

Deep learning-based target tracking with X-ray images for radiotherapy: a narrative review

Predicting pulmonary ventilation damage after radiation therapy for nonsmall cell lung cancer using a ResNet generative adversarial network

Contact Info

Product

Resources

About