Inter/Intra-Observer Agreement in Video-Capsule Endoscopy: Are We Getting It All Wrong? A Systematic Review and Meta-Analysis

Abstract

Video-capsule endoscopy (VCE) reading is a time- and energy-consuming task. Agreement on findings between readers (either different or the same) is a crucial point for increasing performance and providing valid reports. The aim of this systematic review with meta-analysis is to provide an evaluation of inter/intra-observer agreement in VCE reading. A systematic literature search in PubMed, Embase and Web of Science was performed throughout September 2022. The degree of observer agreement, expressed with different test statistics, was extracted. As different statistics are not directly comparable, our analyses were stratified by type of test statistics, dividing them in groups of “None/Poor/Minimal”, “Moderate/Weak/Fair”, “Good/Excellent/Strong” and “Perfect/Almost perfect” to report the proportions of each. In total, 60 studies were included in the analysis, with a total of 579 comparisons. The quality of included studies, assessed with the MINORS score, was sufficient in 52/60 studies. The most common test statistics were the Kappa statistics for categorical outcomes (424 comparisons) and the intra-class correlation coefficient (ICC) for continuous outcomes (73 comparisons). In the overall comparison of inter-observer agreement, only 23% were evaluated as “good” or “perfect”; for intra-observer agreement, this was the case in 36%. Sources of heterogeneity (high, I2 81.8%­98.1%) were investigated with meta-regressions, showing a possible role of country, capsule type and year of publication in Kappa inter-observer agreement. VCE reading suffers from substantial heterogeneity and sub-optimal agreement in both inter- and intra-observer evaluation. Artificial-intelligence-based tools and the adoption of a unified terminology may progressively enhance levels of agreement in VCE reading.

The professional background of a referring physician predicts the diagnostic yield of small bowel capsule endoscopy in suspected small bowel bleeding

“…Another issue relevant to the diagnostic performance of SBCE is interobserver agreement on detection rate and finding classification. A recent metanalysis 21 found that the overall pooled estimates for “perfect” or “good” interobserver and intra-observer agreement were only 23% and 37%, respectively. The reader's experience, accumulating fatigue, and lack of standardization in describing small bowel lesions could affect the interobserver bias 22 23 24 .…”

Robotic Colonoscopy and Beyond: Insights into Modern Lower Gastrointestinal Endoscopy

“…However, although it has been recently shown that CCE is well tolerated, safe, and can reduce the proportion of patients requiring CC [49], another meta-analysis has demonstrated that both bowel cleanliness and complete transit of CCE are insufficient as compared to CC [50]. In addition, it must be acknowledged that a systematic review has demonstrated that the inter/intra-observer agreement of capsule endoscopy is suboptimal, possibly due to heterogeneity in Country, capsule type, and bowel cleansing scale among included studies [51]. In this regard, it has been shown that the use of prucalopride can increase the proportion of patients with complete colon capsule transit and acceptable preparation [52].…”

Clean mucosal area detection of gastroenterologists versus artificial intelligence in small bowel capsule endoscopy

“…However, these judgments tend to be subjective. During colonoscopy, when a gastroenterologist evaluates bowel preparation quality, many instruments, such as the aronchick bowel preparation scale, Boston bowel preparation scale, Ottawa bowel preparation scale, and Chicago bowel preparation scale, are used to reach an objective judgment, [ 6 , 7 ] however, those are still subjective. With the Boston bowel preparation scale, gastrointestinal endoscopists familiarize themselves with photographs of objective indicators of varying degrees of bowel cleanliness and attempt to perform an objective evaluation.…”

“…[ 8 ] However, even if gastroenterologists attempt to establish objective standards, interobserver reliability is diverse and judgments may differ among evaluators. [ 6 , 7 ] Moreover, there has been much less research in the field of CE. Evaluation of the intestinal preparation in CE is mainly performed by setting the visible range of the mucosa.…”