Knowledge‐based quality control of organ delineations in radiation therapy

Abstract

To reduce the likelihood of errors in organ delineations used for radiotherapy treatment planning, a knowledge-based quality control (KBQC) system, which discriminates between valid and anomalous delineations is developed. Method and Materials:The KBQC is comprised of a group-wise inference system and anomaly detection modules trained using historical priors from 296 locally advanced lung and prostate cancer patient computational tomographies (CTs). The inference system discriminates different organs based on shape, relational, and intensity features. For a given delineated image set, the inference system solves a combinatorial optimization problem that results in an organ group whose relational features follow those of the training set considering the posterior probabilities obtained from support vector machine (SVM), discriminant subspace ensemble (DSE), and artificial neural network (ANN) classifiers. These classifiers are trained on nonrelational features with a 10-fold crossvalidation scheme. The anomaly detection module is a bank of ANN autoencoders, each corresponding with an organ, trained on nonrelational features. A heuristic rule detects anomalous organs that exceed predefined organ-specific tolerances for the feature reconstruction error and the classifier's posterior probabilities. Independent data sets with anomalous delineations were used to test the overall performance of the KBQC system. The anomalous delineations were manually manipulated, computer-generated, or propagated based on a transformation obtained by imperfect registrations. Both peer-review-based scoring system and shape similarity coefficient (DSC) were used to label regions of interest (ROIs) as normal or anomalous in two independent test cohorts. Results: The accuracy of the classifiers was ≥99.8%, and the minimum perclass F1-scores were 0.99, 0.99, and 0.98 for SVM, DSE, and ANN, respectively. The group-wise inference system reduced the miss-classification likelihood for the test data set with anomalous delineations compared to each individual classifier and a fused classifier that used the average posterior probability of all classifiers. For 15 independent locally advanced lung patients, the system detected >79% of the anomalous ROIs. For 1320 auto-segmented abdominopelvic organs, the anomaly detection system identified anomalous delineations, which also had low Dice similarity coefficient values with respect to manually delineated organs in the training data set. Conclusion:The KBQC system detected anomalous delineations with superior accuracy compared to classification methods that judge only based on posterior probabilities.

Scalable radiotherapy data curation infrastructure for deep-learning based autosegmentation of organs-at-risk: A case study in head and neck cancer

“…In our view, the curated datasets generated in this study have potentially high value for other applications besides DL-based model training. For example, the expert OARs can be used for: 1) benchmarking existing autosegmentation models/solutions against our defined clinical standards; 2) benchmarking clinician/RO OAR segmentation performance across clinical practice; or 3) creating independent H&N OAR QA tools based on feature extraction and machine learningnot forcibly for autosegmentation but rather for verifying OAR consistency on future clinical H&N planning CTs or used as part of an automated watchdog reviewing output of autosegmentation models (8)(9)(10)36). With the H&N datasets generated in this study, we have already done extensive work on example 1 and begun exploratory work on example 3.…”

“…Artificial Intelligence (AI) promises to address many persistent RT workflow bottlenecks. In the last 25 years, the RT community has witnessed a renaissance in the evaluation and adoption of machine learning (ML)-based solutions in many aspects of RT patient care, from prognostic methods in the patient outcomes research realm including radiomics, to clinical adoption of platforms for automated or semi-automated RT plan generation and related quality assurance (QA) (1)(2)(3)(4)(5)(6)(7)(8)(9)(10). More recently, we have benefited from a virtual explosion in the application of deep convolutional neural networks (hereafter referred to as deep learning, or DL) for tackling complex imaging-related problems.…”

Auto-segmentation for total marrow irradiation

“…It may be possible to link AI algorithms with contour quality assurance using, for example a multi-parametric approach ( 15 , 48 ) or machine learning approach ( 48 ) including sensitivity in Tumor Control Probability (TCP) and Normal Tissue Complication Probability (NTCP) ( 49 ). A recent review article agrees multiple endpoints are needed in assessing contour quality, and clinical validation of meaningful TCP/NTCP endpoints will guide meaningful contour deviations ( 50 ).…”

The future of MRI in radiation therapy: Challenges and opportunities for the MR community

“…Because adapting targets to real‐time changes make clinician approval infeasible, automated QA tools will be needed. Machine learning‐based techniques for detection of erroneous or anomalous delineations 50 or delineation uncertainty maps offer solutions here. 51 , 52 State‐of‐the‐art, online planning assisted with these tools could augment human review in treatment planning and allow smaller treatment margins by reducing inter‐observer variation.…”