Benchmarking definitions of false-positive alerts during computer-aided polyp detection in colonoscopy

Abstract

Introduction The occurrence of false positive (FP) alarms is an important outcome measure in computer-aided colon polyp detection (CADe) studies. However, there is no consensus definition for FPs in clinical trials evaluating CADe in colonoscopy. We aimed to study the diagnostic performance of CADe based on different threshold definitions for FP alarms. Methods A previously validated CADe system was applied to screening/surveillance colonoscopy videos. Different thresholds for FP alerts were defined based on the time an alarm box was continuously traced by the system. Primary outcomes were FP results and specificity using differing FP thresholds. Results 62 colonoscopies were analyzed. CADe specificity and accuracy were 93.2% and 97.8% respectively for a FP threshold definition of > 0.5 seconds, 98.6 % and 99.5% for a FP threshold > 1 second, and 99.8% and 99.9% for a FP threshold > 2 seconds. Conclusion Our analysis demonstrates how different threshold definitions for false positives can impact the reported diagnostic performance of CADe for colon polyp detection.

Computer-Aided Detection False Positives in Colonoscopy

“…This positive correlation raised questions about the conclusion that FPs had no clinical impact [ 12 ]. Another study analyzing FPs also found that a higher number of FPs was associated with longer withdrawal time ( Table 4 ) [ 13 ]. It appears that FPs did contribute to a longer withdrawal time, but that the impact might be quite limited by the commercially available system and experienced endoscopists.…”

“…False positives have various definitions across different studies ( Table 1 ). In general, the term refers to computer prompts indicating polyps that the endoscopist does not consider to be polyps [ 12 , 13 ]. Some investigators have used false alarms to define tracking boxes on non-polyp structures that were continuously tracked [ 7 , 8 ], while other authors ignored brief false positives [ 4 , 12 ].…”

“…However, the threshold required for reporting FPs is unsettled. One report suggested that only FPs > 2 s be reported [ 13 ], and another only reported FPs > 1 s [ 15 ], while the majority of FPs (i.e., more than 90%) lasted <0.5 s [ 13 ]. It is unknown whether ignoring the transient FPs (i.e., those lasting for <1 or 2 s) would increase the risk of missing a real polyp.…”

“…Holzwanger et al defined FPs according to time thresholds (0.5 s, 1 s, and 2 s) [ 13 ]. The different threshold definitions for FPs resulted in different reported diagnostic performances of the CADe, and the data suggested that using the same benchmarks to define FPs is the prerequisite for comparing the performances among different CADe algorithms.…”

“…Optimal bowel preparation is the prerequisite for a high-quality CADe-assisted colonoscopy and is associated with fewer FPs [ 13 ]. As the major source of CADe FP alerts is the wrinkled walls, they can be reduced by ensuring adequate luminal insufflation.…”