Multi-observer concordance and accuracy of the British Thoracic Society scale and other visual assessment qualitative criteria for solid pulmonary nodule assessment using FDG PET-CT

Abstract

To compare the interobserver reliability and diagnostic accuracy of the British Thoracic Society (BTS) scale and other visual assessment criteria in the context of 2-[ 18 F]-fluoro-2deoxy-D-glucose (FDG) positron-emission tomography (PET)-computed tomography (CT) evaluation of solid pulmonary nodules (SPNs).MATERIALS AND METHODS: Fifty patients who underwent FDG PET-CT for assessment of a SPN were identified. Seven reporters with varied experience at four centres graded FDG uptake visually using the British Thoracic Society (BTS) four-point scale. Five reporters also scored SPNs according to three-and five-point visual assessment scales and using semi-quantitative assessment (maximum standardised uptake value [SUV max ]). Interobserver reliability was assessed with the intra-class correlation coefficient (ICC) and weighted Cohen's kappa (k). Diagnostic performance was evaluated by receiver operator characteristic (ROC) analysis.RESULTS: Good interobserver reliability was demonstrated with the BTS scale (ICC¼0.78, 95% confidence interval [CI]: 0.69e0.85) and five-point scale (ICC¼0.78, 95 CI 0.68e0.86), whilst the three-point scale demonstrated moderate reliability (ICC¼0.70, 95% CI: 0.59e0.80). Almost perfect agreement was achieved between two consultants (k¼0.85), and substantial agreement between two other consultants (k¼0.78) using the BTS scale. ROC curves for the BTS and five-point scales demonstrated equivalent accuracy (BTS area under the ROC curve [AUC]¼ 0.768; five-point AUC¼0.768). SUV max was no more accurate compared to the BTS scale (SUV max AUC¼0.794; BTS AUC¼0.768, p¼0.43).