Validation of clinical acceptability of deep-learning-based automated segmentation of organs-at-risk for head-and-neck radiotherapy treatment planning

Abstract

IntroductionOrgan-at-risk segmentation for head and neck cancer radiation therapy is a complex and time-consuming process (requiring up to 42 individual structure, and may delay start of treatment or even limit access to function-preserving care. Feasibility of using a deep learning (DL) based autosegmentation model to reduce contouring time without compromising contour accuracy is assessed through a blinded randomized trial of radiation oncologists (ROs) using retrospective, de-identified patient data.MethodsTwo head and neck expert ROs used dedicated time to create gold standard (GS) contours on computed tomography (CT) images. 445 CTs were used to train a custom 3D U-Net DL model covering 42 organs-at-risk, with an additional 20 CTs were held out for the randomized trial. For each held-out patient dataset, one of the eight participant ROs was randomly allocated to review and revise the contours produced by the DL model, while another reviewed contours produced by a medical dosimetry assistant (MDA), both blinded to their origin. Time required for MDAs and ROs to contour was recorded, and the unrevised DL contours, as well as the RO-revised contours by the MDAs and DL model were compared to the GS for that patient.ResultsMean time for initial MDA contouring was 2.3 hours (range 1.6-3.8 hours) and RO-revision took 1.1 hours (range, 0.4-4.4 hours), compared to 0.7 hours (range 0.1-2.0 hours) for the RO-revisions to DL contours. Total time reduced by 76% (95%-Confidence Interval: 65%-88%) and RO-revision time reduced by 35% (95%-CI,-39%-91%). All geometric and dosimetric metrics computed, agreement with GS was equivalent or significantly greater (p<0.05) for RO-revised DL contours compared to the RO-revised MDA contours, including volumetric Dice similarity coefficient (VDSC), surface DSC, added path length, and the 95%-Hausdorff distance. 32 OARs (76%) had mean VDSC greater than 0.8 for the RO-revised DL contours, compared to 20 (48%) for RO-revised MDA contours, and 34 (81%) for the unrevised DL OARs.ConclusionDL autosegmentation demonstrated significant time-savings for organ-at-risk contouring while improving agreement with the institutional GS, indicating comparable accuracy of DL model. Integration into the clinical practice with a prospective evaluation is currently underway.

Artificial Intelligence–Based Radiotherapy Contouring and Planning to Improve Global Access to Cancer Care

“…These methods promise potential time-savings for target delineation and treatment planning. 13 , 14 Automation, namely deep learning and artificial intelligence (AI) approaches, have been increasingly common in medicine over the past decade, 15 although these have focused on workflows, staffing, and funding in high-income regions.…”

Large-scale dose evaluation of deep learning organ contours in head-and-neck radiotherapy by leveraging existing plans

“…As deep learning-based auto-contouring methods for head-and-neck OARs have been shown to offer satisfactory geometric performance [10] , [6] , the next step is to evaluate their dose impact [11] . However, we observed that dose-based studies on auto-contours tend to use either smaller ( ) [12] , [13] , [14] , [15] , [16] , [17] , [18] or medium-sized ( ) [19] , rather than larger [20] datasets. Studies using larger datasets simply superimpose the automated contours on the clinical dose [20] which does not fully replicate the treatment planning process.…”

“…This work aimed at proposing and assessing an automated plan optimization workflow for retrospective studies that can be easily implemented by clinics due to its use of existing clinical resources. Unlike previous works [12] , [13] , [14] , [15] , [16] , [17] , [18] , we performed this at large-scale and for both photon and proton radiotherapy. To replicate our approach, a clinic can simply use the scripting interface of their treatment planning system (TPS) and convert their planning process into a step-by-step approach.…”

“…Studies using larger datasets simply superimpose the automated contours on the clinical dose [20] which does not fully replicate the treatment planning process. Conversely, studies using smaller or medium-sized test datasets either made manual plans [14] , [17] , [18] , [19] , used knowledge-based planning [13] , a template approach [12] or a priori multi-criteria optimization (MCO) [15] , [16] . Since smaller datasets may be affected by sampling bias, there is a need to perform dose analysis with a larger patient cohort.…”

Knowledge-based quality assurance of a comprehensive set of organ at risk contours for head and neck radiotherapy

“…Standardized and precise organ at risk (OAR) contours are essential for head and neck (HN) radiation therapy, enabling safe treatments and more consistent dose reporting ( 1 ). While manual contouring is time-consuming and prone to user variation, Deep learning (DL) autocontouring methods have demonstrated time savings ( 2 , 3 ) and reduced variation ( 4 , 5 ) compared to manual contouring methods. Autocontouring tools generally perform well, however, a variety of clinically relevant failures, ranging from minor to severe, do occur with no warning given from the model-hosting tool ( Supplementary Figure S1 ).…”