A video based benchmark data set (ENDOTEST) to evaluate computer-aided polyp detection systems

Fitting, Daniel; Krenzer, Adrian; Troya, Joel; Banck, Michael; Sudarevic, Boban; Brand, Markus; Böck, Wolfgang; Wg, Zoller; Rösch, Thomas; Puppe, Frank; Meining, Alexander; Hann, Alexander

doi:10.1080/00365521.2022.2085059

Cited by 14 publications

(7 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the previous study published by our group 23 , EndoMind had a significantly higher number of FPs and a significantly lower FDT, compared to performance in our current work 23 . One reason might be that the EndoMind hardware that was used in the present study infers from every third frame rather than from every single frame, to reduce the number of FPs and to have less delay in the image-processing pipeline.…”

Section: Discussionmentioning

confidence: 99%

Direct comparison of multiple computer-aided polyp detection systems

et al. 2023

Self Cite

View full text Add to dashboard Cite

Background and study aims: Artificial intelligence based computer-aided polyp detection (CADe) systems receive regular updates and occasionally offer customizable detection thresholds which impact their performance, but little is known about these effects. This study aimed to compare the performance of different CADe systems on the same benchmark dataset. Methods: 101 colonoscopy videos were used as benchmark. Each video frame with a visible polyp was manually annotated with bounding boxes resulting in 129,705 polyp images. The videos were then analyzed by three different CADe systems: two versions of GI-Genius, two detection types of EndoAID and the freely-available system EndoMind. The evaluation included an extensive analysis of sensitivity and false-positive rate, among others. Results: EndoAID (Type A), the earlier version of GI-Genius and EndoMind detected all 93 polyps. Both the later version of GI-Genius and EndoAID (Type B) missed one polyp. The mean per-frame sensitivity was of 50.63% and 67.85% for the earlier and the latest version of GI-Genius, 65.60% and 52.95% for EndoAID (Type A and B), and 60.22% for EndoMind. Conclusions: This study compares the performance of different CADe systems, different updates, and different configuration modes. This might help clinicians to select the most appropriate system for their specific needs.

show abstract

Section: Discussionmentioning

confidence: 99%

Direct comparison of multiple computer-aided polyp detection systems

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…DL is an improvement of artificial neural networks, which are composed of more layers of neural networks, allowing the higher layer to contain more abstract information for data prediction. To date, DL has become the leading machine learning tool in the field of computer vision [ 5 , 19 , 33 ]. A typical convolutional neural network (CNN) model used for image processing in DL consists of a series of convolutional networks, including a series of convolutional layers, pooling layers and fully connected layers.…”

Section: Discussionmentioning

confidence: 99%

Deep learning as a novel method for endoscopic diagnosis of chronic atrophic gastritis: a prospective nested case–control study

2022

View full text Add to dashboard Cite

Background and aims Chronic atrophic gastritis (CAG) is a precancerous disease that often leads to the development of gastric cancer (GC) and is positively correlated with GC morbidity. However, the sensitivity of the endoscopic diagnosis of CAG is only 42%. Therefore, we developed a real-time video monitoring model for endoscopic diagnosis of CAG based on U-Net deep learning (DL) and conducted a prospective nested case–control study to evaluate the diagnostic evaluation indices of the model and its consistency with pathological diagnosis. Methods Our cohort consisted of 1539 patients undergoing gastroscopy from December 1, 2020, to July 1, 2021. Based on pathological diagnosis, patients in the cohort were divided into the CAG group or the chronic nonatrophic gastritis (CNAG) group, and we assessed the diagnostic evaluation indices of this model and its consistency with pathological diagnosis after propensity score matching (PSM) to minimize selection bias in the study. Results After matching, the diagnostic evaluation indices and consistency evaluation of the model were better than those of endoscopists [sensitivity (84.02% vs. 62.72%), specificity (97.04% vs. 81.95%), positive predictive value (96.60% vs. 77.66%), negative predictive value (85.86% vs. 68.73%), accuracy rate (90.53% vs. 72.34%), Youden index (81.06% vs. 44.67%), odd product (172.5 vs. 7.64), positive likelihood ratio (28.39 vs. 3.47), negative likelihood ratio (0.16 vs. 0.45), AUC (95% CI) [0.909 (0.884–0.934) vs. 0.740 (0.702–0.778)] and Kappa (0.852 vs. 0.558)]. Conclusions Our prospective nested case–control study proved that the diagnostic evaluation indices and consistency evaluation of the real-time video monitoring model for endoscopic diagnosis of CAG based on U-Net DL were superior to those of endoscopists. Trial registrationChiCTR2100044458, 18/03/2020.

show abstract

“…We call our polyp-detection system ENDOMIND-Advanced. An early, preliminary version of our detection system was experimentally tested and called ENDOMIND [ 56 ]. Nevertheless, ENDOMIND used an early version of YOLOv5 that did not involve our preprocessing, hyperparameter, optimization, and post-processing, and was trained with a smaller dataset.…”

Section: Materials and Methodsmentioning

confidence: 99%

A Real-Time Polyp-Detection System with Clinical Application in Colonoscopy Using Deep Convolutional Neural Networks

et al. 2023

Self Cite

View full text Add to dashboard Cite

Colorectal cancer (CRC) is a leading cause of cancer-related deaths worldwide. The best method to prevent CRC is with a colonoscopy. During this procedure, the gastroenterologist searches for polyps. However, there is a potential risk of polyps being missed by the gastroenterologist. Automated detection of polyps helps to assist the gastroenterologist during a colonoscopy. There are already publications examining the problem of polyp detection in the literature. Nevertheless, most of these systems are only used in the research context and are not implemented for clinical application. Therefore, we introduce the first fully open-source automated polyp-detection system scoring best on current benchmark data and implementing it ready for clinical application. To create the polyp-detection system (ENDOMIND-Advanced), we combined our own collected data from different hospitals and practices in Germany with open-source datasets to create a dataset with over 500,000 annotated images. ENDOMIND-Advanced leverages a post-processing technique based on video detection to work in real-time with a stream of images. It is integrated into a prototype ready for application in clinical interventions. We achieve better performance compared to the best system in the literature and score a F1-score of 90.24% on the open-source CVC-VideoClinicDB benchmark.

show abstract

A video based benchmark data set (ENDOTEST) to evaluate computer-aided polyp detection systems

Cited by 14 publications

References 29 publications

Direct comparison of multiple computer-aided polyp detection systems

Direct comparison of multiple computer-aided polyp detection systems

Deep learning as a novel method for endoscopic diagnosis of chronic atrophic gastritis: a prospective nested case–control study

A Real-Time Polyp-Detection System with Clinical Application in Colonoscopy Using Deep Convolutional Neural Networks

Contact Info

Product

Resources

About