Interobserver Agreement and Correlation of an Automated Algorithm for B‐Line Identification and Quantification With Expert Sonologist Review in a Handheld Ultrasound Device

Abstract

Objectives-B-lines are ultrasound artifacts that can be used to detect a variety of pathologic lung conditions. Computer-aided methods to detect and quantify B-lines may standardize quantification and improve diagnosis by novice users. We sought to test the performance of an automated algorithm for the detection and quantification of B-lines in a handheld ultrasound device (HHUD).Methods-Ultrasound images were prospectively collected on adult emergency department patients with dyspnea. Images from the first 124 patients were used for algorithm development. Clips from 80 unique subjects for testing were randomly selected in a predefined proportion of B-lines (0 B-lines, 1-2 B-lines, 3 or more B-lines) and blindly reviewed by five experts using both a manual and reviewer-adjusted process. Intraclass correlation coefficient (ICC) and weighted kappa were used to measure agreement, while an a priori threshold of an ICC (3,k) of 0.75 and precision of 0.3 were used to define adequate performance.Results-ICC between the algorithm and manual count was 0.84 (95% confidence interval [CI] 0.75-0.90), with a precision of 0.15. ICC between the reviewer-adjusted count and the algorithm count was 0.94 (95% CI 0.90-0.96), and the ICC between the manual and reviewer-adjusted counts was 0.94 (95% CI 0.90-0.96). Weighted kappa was 0.72 (95% CI 0.49-0.95), 0.88 (95% CI 0.74-1), and 0.85 (95% CI 0.89-0.96), respectively.Conclusions-This study demonstrates a high correlation between point-of-care ultrasound experts and an automated algorithm to identify and quantify B-lines using an HHUD. Future research may incorporate this HHUD in clinical studies in multiple settings and users of varying experience levels.

Automated lung ultrasound image assessment using artificial intelligence to identify fluid overload in dialysis patients

“…Recently various algorithms were built for the measurement of B lines in in vivo (32), intensive care unit (12), emergency department ( 9) and patients with dyspnoea (13). The ICC ranged from 0.79-0.94, similar to our ndings (0.892, whereas > 0.75 was considered good performance) (9,10). Unlike our analysis, none of these studies translated into the diagnostic evaluation of uid status or correlated to other objective uid assessments (e.g.…”

“…However, a study using HHUSD versus a high-end ultrasound system (HEUS) to assess B-line count in heart failure patients showed fewer B-lines on HHUSD due to the limited clip store capacity of 2-seconds in HHUSD compared to at least 6 seconds in HEUS (8). We target to standardise the identi cations of B lines; regardless of the level of experience of the staff, the measurement method or the complexity of the device, automated detection by arti cial intelligence (AI) is demonstrated to be feasible and reliable (9,10). Automated B-line detection can be processed by algorithms from deep learning methods (11) and dedicated segmentation, which was shown to be moderately correlated with extracellular lung water (12).…”

Simplified Lung Ultrasound Examination and Telehealth Feasibility in Early SARS-CoV-2 Infection

“…Spatial and temporal resolution and deep learning algorithms have demonstrated variable effects in early studies, having the potential for greater or lesser sensitivity, and are likely bounded by the subjective image search and acquisition process. [45][46][47] Nonetheless, this study has demonstrated that a specific ultrasound examination can be performed during telehealth by patients without any prior training and has future implications for this methodology in other disease states. In COVID-19, the simplified lung examination could be performed not only from home isolation by patients but also in emergency departments, urgent care facilities, or COVID-19 testing centers for outpatients with minimal symptoms.…”

Automated lung ultrasound image assessment using artificial intelligence to identify fluid overload in dialysis patients

“…We also observed similar diagnostic power between physician and AI counts of B lines. Recently various algorithms were built for the measurement of B lines in in vivo [ 34 ], intensive care unit [ 14 ], emergency department [ 11 ] and patients with dyspnoea [ 15 ]. The ICC ranged from 0.79–0.94, similar to our findings (0.892, whereas > 0.75 was considered good performance) [ 11 , 12 ].…”

“…However, a study using HHUSD versus a high-end ultrasound system (HEUS) to assess B-line count in heart failure patients did show fewer B-lines on HHUSD due to the limited clip store capacity of 2-seconds in HHUSD compared to at least 6 seconds in HEUS [ 10 ]. We target to standardise the identifications of B lines; regardless of the level of experience of the staff, the measurement method or the complexity of the device, automated detection by artificial intelligence (AI) is demonstrated to be feasible and reliable [ 11 , 12 ]. Automated B-line detection can be processed by algorithms from deep learning methods [ 13 ] and dedicated segmentation, which was shown to be moderately correlated with extracellular lung water [ 14 ].…”