Markerless Pancreatic Tumor Target Localization Enabled By Deep Learning

Zhao, Wei; Shen, Liyue; Han, Bin; Yang, Yong; Cheng, Kai; Toesca, Diego A.S.; Koong, Albert C.; Chang, Daniel T.; Li, Xing

doi:10.1016/j.ijrobp.2019.05.071

Cited by 63 publications

(80 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our findings indicate that CRNN built upon deep learning algorithms can provide a real‐time solution for tumor tracking in the challenging clinical setting of lung radiotherapy. The localization accuracy of CRNN is similar to what has been reported for prostate and liver although the motion amplitude of lung tumor is larger. CRNN has potential for use in verifying breath‐hold levels, or in gating the image acquisition and treatment delivery with a clinically reasonable tolerance of 2–3 mm.…”

Section: Discussionsupporting

confidence: 83%

“…al. demonstrated that accurate markerless localizations of prostatic and pancreatic tumors are achievable via a convolutional neural network (CNN). While localization was formulated as a registration problem in these studies, the temporal trajectory of tumor in a series of kilovoltage (kV) projections is an extra relevant source of information that should be explored.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Technical Note: 3D localization of lung tumors on cone beam CT projections via a convolutional recurrent neural network

et al. 2020

View full text Add to dashboard Cite

Purpose To design a convolutional recurrent neural network (CRNN) that calculates three‐dimensional (3D) positions of lung tumors from continuously acquired cone beam computed tomography (CBCT) projections, and facilitates the sorting and reconstruction of 4D‐CBCT images. Method Under an IRB‐approved clinical lung protocol, kilovoltage (kV) projections of the setup CBCT were collected in free‐breathing. Concurrently, an electromagnetic signal‐guided system recorded motion traces of three transponders implanted in or near the tumor. Convolutional recurrent neural network was designed to utilize a convolutional neural network (CNN) for extracting relevant features of the kV projections around the tumor, followed by a recurrent neural network for analyzing the temporal patterns of the moving features. Convolutional recurrent neural network was trained on the simultaneously collected kV projections and motion traces, subsequently utilized to calculate motion traces solely based on the continuous feed of kV projections. To enhance performance, CRNN was also facilitated by frequent calibrations (e.g., at 10° gantry rotation intervals) derived from cross‐correlation‐based registrations between kV projections and templates created from the planning 4DCT. Convolutional recurrent neural network was validated on a leave‐one‐out strategy using data from 11 lung patients, including 5500 kV images. The root‐mean‐square error between the CRNN and motion traces was calculated to evaluate the localization accuracy. Result Three‐dimensional displacement around the simulation position shown in the Calypso traces was 3.4 ± 1.7 mm. Using motion traces as ground truth, the 3D localization error of CRNN with calibrations was 1.3 ± 1.4 mm. CRNN had a success rate of 86 ± 8% in determining whether the motion was within a 3D displacement window of 2 mm. The latency was 20 ms when CRNN ran on a high‐performance computer cluster. Conclusions CRNN is able to provide accurate localization of lung tumors with aid from frequent recalibrations using the conventional cross‐correlation‐based registration approach, and has the potential to remove reliance on the implanted fiducials.

show abstract

Section: Discussionsupporting

confidence: 83%

Section: Introductionmentioning

confidence: 99%

Technical Note: 3D localization of lung tumors on cone beam CT projections via a convolutional recurrent neural network

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Deep neural networks have attracted much attention for their ability to learn complex relationships and to incorporate existing knowledge into the inference model through feature extraction and representation learning [20][21][22] . The method has found widespread applications across disciplines, such as computer vision [23][24][25] , autonomous driving 26 , natural language processing 27 , and biomedicine 15,[28][29][30][31][32][33][34][35][36] . Here, we design a hierarchical neural network for Xray CT imaging with ultra-sparse projection views, and develop a structured training process for deep learning to generate three-dimensional (3D) CT images from two-dimensional (2D) X-ray projections.…”

mentioning

confidence: 99%

Patient-specific reconstruction of volumetric computed tomography images from a single projection view via deep learning

2019

Self Cite

View full text Add to dashboard Cite

Tomographic imaging via penetrating waves generates cross-sectional views of the internal anatomy of a living subject. For artefact-free volumetric imaging, projection views from a large number of angular positions are required. Here, we show that a deep-learning model trained to map projection radiographs of a patient to the corresponding 3D anatomy can subsequently generate volumetric tomographic X-ray images of the patient from a single projection view. We demonstrate the feasibility of the approach with upper-abdomen, lung, and head-and-neck computed tomography scans from three patients. Volumetric reconstruction via deep learning could be useful in image-guided interventional procedures such as radiation therapy and needle biopsy, and might help simplify the hardware of tomographic imaging systems. The ability of computed tomography (CT) to take a deep and quantitative look of a patient or an object with high spatial resolution holds significant value in scientific explorations and in medical practice. Traditionally, a tomographic image is obtained via the mathematical inversion of the encoding function of the imaging wave for a given set of measured data from different angular positions (Figs. 1a,b). A prerequisite for artefact-free inversion is the satisfaction of the classical Shannon-Nyquist theorem in angular-data sampling, which Users may view, print, copy, and download text and data-mine the content in such documents, for the purposes of academic research, subject always to the full Conditions of use:

show abstract

“…Rotated CT images with uniformly distributed random angles from −10° to 10° in yaw, pitch, or roll direction are also generated. Additionally, deformation of the pCT image is also introduced, which is achieved by using an existing B‐spline‐based deformable model 44 . Finally, these CT images are filtered by using a variety of Gaussian to reflect possible interfractional changes in the image content.…”

Section: Methodsmentioning

confidence: 99%

“…Additionally, deformation of the pCT image is also introduced, which is achieved by using an existing B-spline-based deformable model. 44 Finally, these CT images are filtered by using a variety of Gaussian to reflect possible interfractional changes in the image content. The filtering is done probabilistically with a standard deviation of the Gaussian distribution selected randomly from 0.1 to 8.…”

Section: B Generation Of Annotated Datasetmentioning

confidence: 99%

A deep learning framework for prostate localization in cone beam CT‐guided radiotherapy

et al. 2020

Self Cite

View full text Add to dashboard Cite

Purpose: To develop a deep learning-based model for prostate planning target volume (PTV) localization on cone beam computed tomography (CBCT) to improve the workflow of CBCT-guided patient setup. Methods: A two-step task-based residual network (T 2 RN) is proposed to automatically identify inherent landmarks in prostate PTV. The input to the T 2 RN is the pretreatment CBCT images of the patient, and the output is the deep learning-identified landmarks in the PTV. To ensure robust PTV localization, the T 2 RN model is trained by using over thousand sets of CT images with labeled landmarks, each of the CTs corresponds to a different scenario of patient position and/or anatomy distribution generated by synthetically changing the planning CT (pCT) image. The changes, including translation, rotation, and deformation, represent vast possible clinical situations of anatomy variations during a course of radiation therapy (RT). The trained patient-specific T 2 RN model is tested by using 240 CBCTs from six patients. The testing CBCTs consists of 120 original CBCTs and 120 synthetic CBCTs. The synthetic CBCTs are generated by applying rotation/translation transformations to each of the original CBCT. Results: The systematic/random setup errors between the model prediction and the reference are found to be <0.25/2.46 mm and 0.14/1.41°in translation and rotation dimensions, respectively. Pearson's correlation coefficient between model prediction and the reference is higher than 0.94 in translation and rotation dimensions. The Bland-Altman plots show good agreement between the two techniques. Conclusions: A novel T 2 RN deep learning technique is established to localize the prostate PTV for RT patient setup. Our results show that highly accurate marker-less prostate setup is achievable by leveraging the state-of-the-art deep learning strategy.

show abstract

Markerless Pancreatic Tumor Target Localization Enabled By Deep Learning

Cited by 63 publications

References 35 publications

Technical Note: 3D localization of lung tumors on cone beam CT projections via a convolutional recurrent neural network

Technical Note: 3D localization of lung tumors on cone beam CT projections via a convolutional recurrent neural network

Patient-specific reconstruction of volumetric computed tomography images from a single projection view via deep learning

A deep learning framework for prostate localization in cone beam CT‐guided radiotherapy

Contact Info

Product

Resources

About