Chest radiograph reading panel performance in a Bangladesh pneumococcal vaccine effectiveness study

Abstract

IntroductionTo evaluate WHO chest radiograph interpretation processes during a pneumococcal vaccine effectiveness study of children aged 3–35 months with suspected pneumonia in Sylhet, Bangladesh.MethodsEight physicians masked to all data were standardised to WHO methodology and interpreted chest radiographs between 2015 and 2017. Each radiograph was randomly assigned to two primary readers. If the primary readers were discordant for image interpretability or the presence or absence of primary endpoint pneumonia (PEP), then another randomly selected, masked reader adjudicated the image (arbitrator). If the arbitrator disagreed with both primary readers, or concluded no PEP, then a masked expert reader finalised the interpretation. The expert reader also conducted blinded quality control (QC) for 20% of randomly selected images. We evaluated agreement between primary readers and between the expert QC reading and the final panel interpretation using per cent agreement, unadjusted Cohen’s kappa, and a prevalence and bias-adjusted kappa.ResultsAmong 9723 images, the panel classified 21.3% as PEP, 77.6% no PEP and 1.1% uninterpretable. Two primary readers agreed on interpretability for 98% of images (kappa, 0.25; prevalence and bias-adjusted kappa, 0.97). Among interpretable radiographs, primary readers agreed on the presence or absence of PEP in 79% of images (kappa, 0.35; adjusted kappa, 0.57). Expert QC readings agreed with final panel conclusions on the presence or absence of PEP for 92.9% of 1652 interpretable images (kappa, 0.75; adjusted kappa, 0.85).ConclusionPrimary reader performance and QC results suggest the panel effectively applied the WHO chest radiograph criteria for pneumonia.

Training physicians in India to interpret pediatric chest radiographs according to World Health Organization research methodology

“…The World Health Organization (WHO) methodology for classifying pediatric chest radiographs, developed in 1997, has been applied to efficacy and effectiveness studies of bacterial conjugate vaccines as well as more recently to epidemiological research of pediatric lower respiratory infections [7][8][9][10][11][12][13][14][15][16]. This methodology aims to identify chest radiographs that have a higher probability of bacterial etiology, with an emphasis on Streptococcus pneumoniae and Haemophilus influenzae, and prioritizes both interobserver reliability and specificity [8,15].…”

Performance of lung ultrasound in the diagnosis of pediatric pneumonia in Mozambique and Pakistan

“…Higher interoperator and interrater reliability for LUS interpretation than for CXR interpretation in identifying pediatric pneumonia is supported by the literature (Figure 4). 14,16–33 We contrasted kappas observed in this study with kappas observed in the literature among other LUS and CXR interpreters. Kappas between LUS interpreters were 0.900 (in Pakistan) and 0.917 (in Mozambique) in this study (expert LUS interpreters) and ranged from 0.635 to 0.930 in the literature, whereas kappa between CXR interpreters ranged from −0.04 to 0.62 in this study and from 0.35 to 0.74 in the literature.…”

Design and conduct of facility-based surveillance for severe childhood pneumonia in the Household Air Pollution Intervention Network (HAPIN) trial

“…Instead, we are using chest radiography in India, and applying rigorous methodology developed and validated by the WHO CRES working group [34]. Our group has substantial experience using the WHO chest radiograph interpretation, and we will utilise chest radiography in Guatemala, Peru, and Rwanda when lung ultrasound is not obtained [35].…”