Adaptive weighted log subtraction based on neural networks for markerless tumor tracking using dual‐energy fluoroscopy

Haytmyradov, M.; Mostafavi, Hassan; Cassetta, Roberto; Patel, Rakesh; Sürücü, Murat; Zhu, Liangjia; Roeske, John C.

doi:10.1002/mp.13941

Cited by 4 publications

(3 citation statements)

References 21 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 2 , 8 , 9 From the image processing viewpoint, we can distinguish the difficulties into four important factors: Obstacle overlapping: For example, high‐contrast bone features projected on XF cause false tracking. These obstacles require to be suppressed 10 , 11 , 12 , 13 or recognized as unimportant (ignored). 14 Poor visibility: Because tumor contrast in XF is usually insufficient, the tumor position should be estimated by surrounding structures that may be more visible or the motion should be enhanced using XF subtraction.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Explainability and controllability of patient‐specific deep learning with attention‐based augmentation for markerless image‐guided radiotherapy

Terunuma

Sakae

et al. 2022

Medical Physics

View full text Add to dashboard Cite

Background We reported the concept of patient‐specific deep learning (DL) for real‐time markerless tumor segmentation in image‐guided radiotherapy (IGRT). The method was aimed to control the attention of convolutional neural networks (CNNs) by artificial differences in co‐occurrence probability (CoOCP) in training datasets, that is, focusing CNN attention on soft tissues while ignoring bones. However, the effectiveness of this attention‐based data augmentation has not been confirmed by explainable techniques. Furthermore, compared to reasonable ground truths, the feasibility of tumor segmentation in clinical kilovolt (kV) X‐ray fluoroscopic (XF) images has not been confirmed. Purpose The first aim of this paper was to present evidence that the proposed method provides an explanation and control of DL behavior. The second purpose was to validate the real‐time lung tumor segmentation in clinical kV XF images for IGRT. Methods This retrospective study included 10 patients with lung cancer. Patient‐specific and XF angle‐specific image pairs comprising digitally reconstructed radiographs (DRRs) and projected‐clinical‐target‐volume (pCTV) images were calculated from four‐dimensional computer tomographic data and treatment planning information. The training datasets were primarily augmented by random overlay (RO) and noise injection (NI): RO aims to differentiate positional CoOCP in soft tissues and bones, and NI aims to make a difference in the frequency of occurrence of local and global image features. The CNNs for each patient‐and‐angle were automatically optimized in the DL training stage to transform the training DRRs into pCTV images. In the inference stage, the trained CNNs transformed the test XF images into pCTV images, thus identifying target positions and shapes. Results The visual analysis of DL attention heatmaps for a test image demonstrated that our method focused CNN attention on soft tissue and global image features rather than bones and local features. The processing time for each patient‐and‐angle‐specific dataset in the training stage was ∼30 min, whereas that in the inference stage was 8 ms/frame. The estimated three‐dimensional 95 percentile tracking error, Jaccard index, and Hausdorff distance for 10 patients were 1.3–3.9 mm, 0.85–0.94, and 0.6–4.9 mm, respectively. Conclusions The proposed attention‐based data augmentation with both RO and NI made the CNN behavior more explainable and more controllable. The results obtained demonstrated the feasibility of real‐time markerless lung tumor segmentation in kV XF images for IGRT.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Obstacle overlapping: For example, high‐contrast bone features projected on XF cause false tracking. These obstacles require to be suppressed 10 , 11 , 12 , 13 or recognized as unimportant (ignored). 14 …”

Section: Introductionmentioning

confidence: 99%

Explainability and controllability of patient‐specific deep learning with attention‐based augmentation for markerless image‐guided radiotherapy

Terunuma

Sakae

et al. 2022

Medical Physics

View full text Add to dashboard Cite

show abstract

“…In the work by Haytmyradov et al (2020) material selection weighting factors were obtained by using a calibration phantom together with the convolutional neural network. The resulting DE images showed better material selection, however, noise suppression was not addressed in this study.…”

Section: Introductionmentioning

confidence: 99%

Adaptive dual-energy algorithm based on pre-calibrated weighting factors for chest radiography

Romadanov¹,

Abeywardhana²,

Sattarivand³

2022

Phys. Med. Biol.

View full text Add to dashboard Cite

Objective: To develop a dual-energy (DE) algorithm with spatially varying weighting factors for material selection and noise suppression. Approach: Calibration step-phantoms, with overlapping slabs of solid water and bone with different thicknesses, were used to obtain the pre-calibrated material selection and noise reduction weighting factors. The Material selection weighting factors were calculated by finding a zero of contrast-to-noise-ratio (CNR) between regions with two overlapping materials and regions of only target material, while noise suppression weighting factors were determined by maximizing signal-to-noise ratio for overlapping regions. The pre-calibrated weighting factors were fitted with low and high energy radiograph of two Rando phantoms to create maps of material selection and noise suppression weighting factors, which used with duel energy (DE) algorithm and anti-correlated noise reduction (ACNR) algorithm to generate DE images. Three different implementations, including two different sizes of Rando phantoms and two different orientations (oblique and Anterior-Posterior), were investigated. Soft-tissue and bone only images of Rando phantoms were obtained with five combination of DE algorithms and CNR, contrast, and noise values of selected regions of interest (ROIs) were compared to evaluate the performance of the novel method: simple log subtraction (SLS), SLS with uniform ACNR, adaptive DE (aDE), aDE with uniform ACNR, and aDE & adaptive ACNR (aACNR). Main results: Compared to SLS, the aDE algorithm demonstrated improved image quality in all three orientations. CNR increased with better contrast for both soft-tissue and bone images. Implementation of aACNR algorithm resulted in further reduction of image noise and improvements in CNR at the cost of contrast. However, aACNR algorithm showed better contrast comparing to ACNR method. Significance: A novel DE algorithm was proposed, which showed improved material selection and noise suppression as compared to the conventional DE techniques and can be easily implemented in a clinical environment for real-time DE image generation.

show abstract

Deep Learning Methods for Image Guidance in Radiation Therapy

Paysan

Roggen

Zhu

et al. 2020

Artificial Neural Networks in Pattern Recognition

View full text Add to dashboard Cite

Adaptive weighted log subtraction based on neural networks for markerless tumor tracking using dual‐energy fluoroscopy

Cited by 4 publications

References 21 publications

Explainability and controllability of patient‐specific deep learning with attention‐based augmentation for markerless image‐guided radiotherapy

Explainability and controllability of patient‐specific deep learning with attention‐based augmentation for markerless image‐guided radiotherapy

Adaptive dual-energy algorithm based on pre-calibrated weighting factors for chest radiography

Deep Learning Methods for Image Guidance in Radiation Therapy

Contact Info

Product

Resources

About