The impact of inter-observer variation in delineation on robustness of radiomics features in non-small cell lung cancer

Abstract

Artificial intelligence and radiomics have the potential to revolutionise cancer prognostication and personalised treatment. Manual outlining of the tumour volume for extraction of radiomics features (RF) is a subjective process. This study investigates robustness of RF to inter-observer variation (IOV) in contouring in lung cancer. We utilised two public imaging datasets: ‘NSCLC-Radiomics’ and ‘NSCLC-Radiomics-Interobserver1’ (‘Interobserver’). For ‘NSCLC-Radiomics’, we created an additional set of manual contours for 92 patients, and for ‘Interobserver’, there were five manual and five semi-automated contours available for 20 patients. Dice coefficients (DC) were calculated for contours. 1113 RF were extracted including shape, first order and texture features. Intraclass correlation coefficient (ICC) was computed to assess robustness of RF to IOV. Cox regression analysis for overall survival (OS) was performed with a previously published radiomics signature. The median DC ranged from 0.81 (‘NSCLC-Radiomics’) to 0.85 (‘Interobserver’—semi-automated). The median ICC for the ‘NSCLC-Radiomics’, ‘Interobserver’ (manual) and ‘Interobserver’ (semi-automated) were 0.90, 0.88 and 0.93 respectively. The ICC varied by feature type and was lower for first order and gray level co-occurrence matrix (GLCM) features. Shape features had a lower median ICC in the ‘NSCLC-Radiomics’ dataset compared to the ‘Interobserver’ dataset. Survival analysis showed similar separation of curves for three of four RF apart from ‘original_shape_Compactness2’, a feature with low ICC (0.61). The majority of RF are robust to IOV, with first order, GLCM and shape features being the least robust. Semi-automated contouring improves feature stability. Decreased robustness of a feature is significant as it may impact upon the features’ prognostic capability.

Clinical validation of a semi‐automated segmentation algorithm for target volume definition on planning CT and CBCT in stereotactic body radiotherapy (SBRT) for peripheral lung lesions

“…Subsequently, this could result in undermining tumour control, as we emphasised in the current study or increasing toxicities 17,18 . Also, interobserver variability could impact the robustness of the radiomics features negatively, and the standardisation of target volume delineation could improve the stability of the prognostic value of the radiomics features 19 …”

“… 17 , 18 Also, interobserver variability could impact the robustness of the radiomics features negatively, and the standardisation of target volume delineation could improve the stability of the prognostic value of the radiomics features. 19 …”

“…17,18 Also, interobserver variability could impact the robustness of the radiomics features negatively, and the standardisation of target volume delineation could improve the stability of the prognostic value of the radiomics features. 19 In the current study, we introduced a semi-automated delineation tool for target volume delineation based on the change of HU units to define the tumour boundaries using a function available in the TPS Pinnacle, 3 which is easily applicable in the clinical environment within the TPS. This semi-automated delineation tool is a simple approach that uses the HUs in the lesion and the surrounding tissue to define the lower threshold, which makes it more patient-specific than other approaches using a fixed HU Threshold, 21 therefore, can be used for P-CT datasets and CBCT datasets.…”

The role of artificial intelligence based on PET/CT radiomics in NSCLC: Disease management, opportunities, and challenges

“…Some features depend on different segmentation methods, which also affects the reproducibility and repeatability of radiomics. The traditional method is manual segmentation, which is undoubtedly time-consuming and labor-dependent, and variability between and within observers is inevitable (87). In recent years, automatic and semi-automatic segmentation techniques have been recommended to improve efficiency and reproducibility (88).…”

Prediction of 177Lu-DOTATATE Therapy Outcomes in Neuroendocrine Tumor Patients Using Semi-Automatic Tumor Delineation on 68Ga-DOTATATE PET/CT

“…Despite the limitations, the semi-automatic tumor delineation method yielded meaningful prognostic information in our patient cohort and the benefits of using semi-automated tumor delineation on efficiency and standardization are well acknowledged in the literature. For example, previous studies showed that inter-observer variability in tumor contouring is a potential barrier to radiomic analysis of oncologic PET, and semi-automatic tumor delineation reduced the variability [ 42 , 43 ]. Each PET exam can be analyzed in as quickly as 5 min with semi-automatic tumor contouring [ 18 ], consistent with our experience in the present study.…”