First Report On Physician Assessment and Clinical Acceptability of Custom-Retrained Artificial Intelligence Models for Clinical Target Volume and Organs-at-Risk Auto-Delineation for Postprostatectomy Patients

Hobbis, Dean; Mund, Karl; Duan, Jingwei; Rwigema, Jean‐Claude M.; Wong, William W.; Schild, Steven E.; Keole, Sameer R.; Feng, Xue; Chen, Quan; Vargas, Carlos; Rong, Yi

doi:10.1016/j.prro.2023.03.011

Cited by 13 publications

(14 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Segmentation of healthy organs from Computed Tomography (CT) images is critical and beneficial in a number of applications, including the generation of anthropomorphic computational models, delimitation of organs at risk in radiation therapy (RT) treatment planning (1–4), and other kinds of computer-assisted applications, such as pathologic detection (5, 6), prognosis and outcome prediction (7, 8), image quantification (9, 10), and radiation dosimetry calculations (11–13). The manual slice-by-slice segmentation of organs can be labor-intensive and time-consuming, in addition to the high inter- and intra-observer variability reported for segmentation of healthy organs and malignant lesions (14, 15). Since the emergence of machine learning and deep learning (DL) algorithms in medical imaging research, especially medical image segmentation, a number of studies focused on automatic segmentation of structures from CT images and other imaging modalities (16–18).…”

Section: Introductionmentioning

confidence: 99%

“…Xu et al (24) focused on the occurrence of outliers during image segmentation and how to solve this problem. Recent studies addressed the limitations and benefits of DL-based organ segmentation in real-life clinical scenarios (14, 25). The comparison of the results achieved by different techniques using private/local databases is not straightforward given that the used datasets are not publicly available.…”

Section: Introductionmentioning

confidence: 99%

“…The comparison of the results achieved by different techniques using private/local databases is not straightforward given that the used datasets are not publicly available. Besides, it’s well established that acquisition, scanner, and demographic parameters can affect the performance of a model on external unseen datasets from other centers (14, 26). Ma et al (19) described the low performance of segmentation models trained and inferenced on different databases for abdominal organs segmentation task.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep learning-assisted multiple organ segmentation from whole-body CT images

Salimi,

Shiri,

Mansouri

et al. 2023

Preprint

View full text Add to dashboard Cite

Background: Automated organ segmentation from computed tomography (CT) images facilitates a number of clinical applications, including clinical diagnosis, monitoring of treatment response, quantification, radiation therapy treatment planning, and radiation dosimetry. Purpose: To develop a novel deep learning framework to generate multi-organ masks from CT images for 23 different body organs. Methods: A dataset consisting of 3106 CT images (649,398 axial 2D CT slices, 13,640 images/segment pairs) and ground-truth manual segmentation from various online available databases were collected. After cropping them to body contour, they were resized, normalized and used to train separate models for 23 organs. Data were split to train (80%) and test (20%) covering all the databases. A Res-UNET model was trained to generate segmentation masks from the input normalized CT images. The model output was converted back to the original dimensions and compared with ground-truth segmentation masks in terms of Dice and Jaccard coefficients. The information about organ positions was implemented during post-processing by providing six anchor organ segmentations as input. Our model was compared with the online available TotalSegmentator model through testing our model on their test datasets and their model on our test datasets. Results: The average Dice coefficient before and after post-processing was 84.28% and 83.26% respectively. The average Jaccard index was 76.17 and 70.60 before and after post-processing respectively. Dice coefficients over 90% were achieved for the liver, heart, bones, kidneys, spleen, femur heads, lungs, aorta, eyes, and brain segmentation masks. Post-processing improved the performance in only nine organs. Our model on the TotalSegmentator dataset was better than their models on our dataset in five organs out of 15 common organs and achieved almost similar performance for two organs. Conclusions: The availability of a fast and reliable multi-organ segmentation tool leverages implementation in clinical setting. In this study, we developed deep learning models to segment multiple body organs and compared the performance of our models with different algorithms. Our model was trained on images presenting with large variability emanating from different databases producing acceptable results even in cases with unusual anatomies and pathologies, such as splenomegaly. We recommend using these algorithms for organs providing good performance. One of the main merits of our proposed models is their lightweight nature with an average inference time of 1.67 seconds per case per organ for a total-body CT image, which facilitates their implementation on standard computers.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning-assisted multiple organ segmentation from whole-body CT images

Salimi,

Shiri,

Mansouri

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…It is highly possible that the contour labels from different ROs may vary due to their distinct training backgrounds, experiences, personal preferences, and other factors. 26,27 The inherent bias in human labels, on the other hand, can partially affect the selection of z values, which determine the pass criteria. When multiple ROs collaborate to define acceptability, some regions of the contour may demonstrate larger tolerances, with the acceptable deviation increasing to accommodate the varying perspectives of the ROs.…”

Section: Discussionmentioning

confidence: 99%

Contour subregion error detection methodology using deep learning auto‐segmentation

Duan,

Bernard,

Rong

et al. 2023

Medical Physics

Self Cite

View full text Add to dashboard Cite

BackgroundInaccurate manual organ delineation is one of the high‐risk failure modes in radiation treatment. Numerous automated contour quality assurance (QA) systems have been developed to assess contour acceptability; however, manual inspection of flagged cases is a time‐consuming and challenging process, and can lead to users overlooking the exact error location.PurposeOur aim is to develop and validate a contour QA system that can effectively detect and visualize subregional contour errors, both qualitatively and quantitatively.Methods/MaterialsA novel contour subregion error detection (CSED) system was developed using subregional surface distance discrepancies between manual and deep learning auto‐segmentation (DLAS) contours. A validation study was conducted using a head and neck public dataset containing 339 cases and evaluated according to knowledge‐based pass criteria derived from a clinical training dataset of 60 cases. A blind qualitative evaluation was conducted, comparing the results from the CSED system with manual labels. Subsequently, the CSED‐flagged cases were re‐examined by a radiation oncologist.ResultsThe CSED system could visualize the diverse types of subregional contour errors qualitatively and quantitatively. In the validation dataset, the CSED system resulted in true positive rates (TPR) of 0.814, 0.800, and 0.771; false positive rates (FPR) of 0.310, 0.267, and 0.298; and accuracies of 0.735, 0.759, and 0.730, for brainstem and left and right parotid contours, respectively. The CSED‐assisted manual review caught 13 brainstem, 19 left parotid, and 21 right parotid contour errors missed by conventional human review. The TPR/FPR/accuracy of the CSED‐assisted manual review improved to 0.836/0.253/0.784, 0.831/0.171/0.830, and 0.808/0.193/0.807 for each structure, respectively. Further, the time savings achieved through CSED‐assisted review improved by 75%, with the time for review taking 24.81 ± 12.84, 26.75 ± 10.41, and 28.71 ± 13.72 s for each structure, respectively.ConclusionsThe CSED system enables qualitative and quantitative detection, localization, and visualization of manual segmentation subregional errors utilizing DLAS contours as references. The use of this system has been shown to help reduce the risk of high‐risk failure modes resulting from inaccurate organ segmentation.

show abstract

“…However, relevant studies on their clinical implementation is quite limited. Previous studies by Duan et al [13] and Hobbis et al [18] plans were reviewed by an internal panel before approved for clinical treatment.…”

Section: Introductionmentioning

confidence: 99%

Enhancing Precision in Rectal Cancer Radiotherapy: Localized Fine-Tuning of Deep-learning based Auto-segmentation (DLAS) Model for Clinical Target Volume and Organs-at-risk

Geng,

Sui,

et al. 2024

Preprint

View full text Add to dashboard Cite

Background and Purpose Various deep learning auto-segmentation (DLAS) models have been proposed, some of which commercialized. However, the issue of performance degradation is notable when pretrained models are deployed in the clinic. This study aims to enhance precision of a popular commercial DLAS product in rectal cancer radiotherapy by localized fine-tuning, addressing challenges in practicality and generalizability in real-world clinical settings. Materials and Methods A total of 120 Stage II/III mid-low rectal cancer patients were retrospectively enrolled and divided into three datasets: training (n = 60), external validation (ExVal, n = 30), and generalizability evaluation (GenEva, n = 30) datasets respectively. The patients in the training and ExVal dataset were acquired on the same CT simulator, while those in GenEva were on a different CT simulator. The commercial DLAS software was first localized fine-tuned for clinical target volume (CTV) and organs-at-risk (OAR) using the training data, and then validated on ExVal and GenEva respectively. Performance evaluation involved comparing the localized fine-tuned model (LFT) with the vendor-provided pretrained model (VPM) against ground truth contours, using metrics like Dice similarity coefficient (DSC), 95th Hausdorff distance (95HD), sensitivity and specificity. Results Localized fine-tuning significantly improved CTV delineation accuracy (p < 0.05) with LFT outperforming VPM in target volume, DSC, 95HD and specificity. Both models exhibited adequate accuracy for bladder and femoral heads, and LFT demonstrated significant enhancement in segmenting the more complex small intestine. We did not identify performance degradation when LFT and VPM models were applied in the GenEva dataset. Conclusions The necessity and potential benefits of localized fine-tuning DLAS towards institution-specific model adaption is underscored. The commercial DLAS software exhibits superior accuracy once localized fine-tuned, and is highly robust to imaging equipment changes.

show abstract

First Report On Physician Assessment and Clinical Acceptability of Custom-Retrained Artificial Intelligence Models for Clinical Target Volume and Organs-at-Risk Auto-Delineation for Postprostatectomy Patients

Cited by 13 publications

References 48 publications

Deep learning-assisted multiple organ segmentation from whole-body CT images

Deep learning-assisted multiple organ segmentation from whole-body CT images

Contour subregion error detection methodology using deep learning auto‐segmentation

Enhancing Precision in Rectal Cancer Radiotherapy: Localized Fine-Tuning of Deep-learning based Auto-segmentation (DLAS) Model for Clinical Target Volume and Organs-at-risk

Contact Info

Product

Resources

About