Interobserver Agreement of Lung Ultrasound Findings of COVID‐19

Abstract

Background Lung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID‐19. Previously described LUS manifestations for COVID‐19 include B‐lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID‐19 is unknown. Methods This study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans ( n = 180 independent observations) collected from patients with COVID‐19, diagnosed via RT‐PCR. These studies were randomly selected from an image database consisting of COVID‐19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values ( κ ) were used to calculate IRR. Results There was substantial IRR on the following items: normal LUS scan ( κ = 0.79 [95% CI: 0.72–0.87]), presence of B‐lines ( κ = 0.79 [95% CI: 0.72–0.87]), ≥3 B‐lines observed ( κ = 0.72 [95% CI: 0.64–0.79]). Moderate IRR was observed for the presence of any consolidation ( κ = 0.57 [95% CI: 0.50–0.64]), subpleural consolidation ( κ = 0.49 [95% CI: 0.42–0.56]), and presence of effusion ( κ = 0.49 [95% CI: 0.41–0.56]). Fair IRR was observed for pleural thickening ( κ = 0.23 [95% CI: 0.15–0.30]). Discussion Many LUS manifestations for COVID‐19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID‐19 may include the presence/count of B‐lines or determining if a scan is normal. Clinical protocols for LUS with COVID‐19 may require additional observers for the confirmation of less reliable findings such as consolidations.

Point-of-Care Ultrasound (POCUS) Predicts Clinical Outcomes in Patients with COVID-19

“…33 Several of the findings (especially B-lines) have excellent interrater reliability and can be expediently learned. 30,34,35 Based on the results of this study, a rapid assessment of the anterior lungs may provide sufficient risk stratification and warrants further study.…”

“…We found that LUS has moderate-tosubstantial IRR for LUS among COVID-19 for the findings included in this study. 30 Based on our IRR findings, we developed an interpretation protocol. First, two researchers would independently apply the consensus guidelines (Appendix) to create a standardized approach to image interpretation.…”

Efficacy of a short course of lung ultrasound for primary care physicians in the assessment of COVID-19-positive patients

“…The interrater agreement in detecting sub-pleural consolidations and pleural thickening resulted lower but substantial. 9 However, LUS was performed by trained medical doctors in an inpatient setting.…”

Point-of-care lung ultrasound in COVID-19 patients: inter- and intra-observer agreement in a prospective observational study

“…In a recently published study investigating interobserver agreement of LUS in COVID-19 (although without measuring intraobserver agreement), Kumar et al found lower agreement for consolidations, similar agreement for pleural thickening and higher agreement especially when assessing B-lines 18 . This is in line with our assumption that low agreement in B-Line quantification might be due to a lack in specific training as observers met for a one hour calibration session before grading in the aforementioned study.…”