International Multi-Specialty Expert Physician Preoperative Identification of Extranodal Extension in Oropharyngeal Cancer Patients using Computed Tomography: Prospective Blinded Human Inter-Observer Performance Evaluation

Abstract

Background: Extranodal extension (ENE) is an important adverse prognostic factor in oropharyngeal cancer (OPC) and is often employed in therapeutic decision making. Clinician-based determination of ENE from radiological imaging is a difficult task with high inter-observer variability. However, the role of clinical specialty on the determination of ENE has been unexplored. Methods: Pre-therapy computed tomography (CT) images for 24 human papillomavirus-positive (HPV+) OPC patients were selected for the analysis; 6 scans were randomly chosen to be duplicated, resulting in a total of 30 scans of which 21 had pathologically-confirmed ENE. 34 expert clinician annotators, comprised of 11 radiologists, 12 surgeons, and 11 radiation oncologists separately evaluated the 30 CT scans for ENE and noted the presence or absence of specific radiographic criteria and confidence in their prediction. Discriminative performance was measured using accuracy, sensitivity, specificity, area under the receiver operating characteristic curve (AUC), and Brier score for each physician. Statistical comparisons of discriminative performance were calculated using Mann Whitney U tests. Significant radiographic factors in correct discrimination of ENE status were determined through a logistic regression analysis. Interobserver agreement was measured using Fleiss kappa. Results: The median accuracy for ENE discrimination across all specialties was 0.57. There were significant differences between radiologists and surgeons for Brier score (0.33 vs. 0.26), radiation oncologists and surgeons for sensitivity (0.48 vs. 0.69), and radiation oncologists and radiologists/surgeons for specificity (0.89 vs. 0.56). There were no significant differences between specialties for accuracy or AUC. Indistinct capsular contour, nodal necrosis, and nodal matting were significant factors in regression analysis. Fleiss kappa was less than 0.6 for all the radiographic criteria, regardless of specialty. Conclusions: Detection of ENE in HPV+OPC patients on CT imaging remains a difficult task with high variability, regardless of clinician specialty. Although some differences do exist between the specialists, they are often minimal. Further research in automated analysis of ENE from radiographic images is likely needed.

Artificial intelligence across oncology specialties: current applications and emerging tools

“…67 The presence of extranodal extension (ENE) in neck nodes portends more aggressive cancer, though it is very challenging to detect on imaging. 68 Kann et al developed and validated a deep CNN model to predict pathologic ENE on CT imaging 69 and used ECOG-ACRIN E3311 data in a quality improvement study to suggest better ability to predict pathologic ENE compared with head and neck radiologists. 70 Genomics-driven precision oncology Several categories of precision oncology using clinicogenomic data are emerging, ranging from improving prognostication to biomarker selection to drug development.…”

Extranodal extension in head and neck squamous cell carcinoma: need for accurate pretherapeutic staging to select optimum treatment and minimize toxicity

“…A multispecialty expert physicians’ group [74 ▪ ] in an observation performance evaluation study was asked to identify the presence of rENE+ in CT of HPV and OPSCC patients. The median accuracy of rENE and discrimination was 0.57 with significant differences between radiologists and surgeons for Brier score (0.33 vs. 0.26), radiation oncologists/surgeons for sensitivity (0.48 vs. 0.69), and radiation oncologists and radiologists/surgeons for specificity (0.89 vs. 0.56).…”

Prediction of extranodal extension in head and neck squamous cell carcinoma by CT images using an evolutionary learning model

“…Contrast-enhanced computed tomography (CT) scan is the most widely used method to predict ENE status for HNSCC patients in clinical practice. However, the literature revealed that this method has limited diagnostic performance, with reported sensitivity ranging from 43.7 to 69% and the area under the receiver operating characteristic curve (AUC) ranging from 0.6 to 0.69 [ 9 – 13 ]. Furthermore, high inter-observer variability is also reported [ 9 , 11 – 13 ].…”