No abstract

BACKGROUND AND AIMS: Endoscopic disease activity scoring in ulcerative colitis (UC) is useful in clinical practice but done infrequently. It is required in clinical trials, where it is expensive and slow because human central readers are needed. A machine learning algorithm automating the process could elevate clinical care and facilitate clinical research. Prior work using single-institution databases and endoscopic still images has been promising. METHODS: Seven hundred and ninety-five full-length endoscopy videos were prospectively collected from a phase 2 trial of mirikizumab with 249 patients from 14 countries, totaling 19.5 million image frames. Expert central readers assigned each full-length endoscopy videos 1 endoscopic Mayo score (eMS) and 1 Ulcerative Colitis Endoscopic Index of Severity (UCEIS) score. Initially, video data were cleaned and abnormality features extracted using convolutional neural networks. Subsequently, a recurrent neural network was trained on the features to predict eMS and UCEIS from individual full-length endoscopy videos. RESULTS: The primary metric to assess the performance of the recurrent neural network model was quadratic weighted kappa (QWK) comparing the agreement of the machine-read endoscopy score with the human central reader score. QWK progressively penalizes disagreements that exceed 1 level. The model's agreement metric was excellent, with a QWK of 0.844 (95% confidence interval, 0.787-0.901) for eMS and 0.855 (95% confidence interval, 0.80-0.91) for UCEIS. CONCLUSIONS: We found that a deep learning algorithm can be trained to predict levels of UC severity from full-length endoscopy videos. Our data set was prospectively collected in a multinational clinical trial, videos rather than still images were used, UCEIS and eMS were reported, and machine learning algorithm performance metrics met or exceeded those previously published for UC severity scores.

Introduction-Reliable in situ diagnosis of diminutive (≤ 5 mm) colorectal polyps could allow for "resect and discard" and "diagnose and leave" strategies resulting in one billion dollars in cost savings per year in the U.S. alone. Current methodologies have failed to consistently meet the Preservation and Incorporation of Valuable endoscopic Innovations (PIVI) initiative thresholds. Convolutional neural networks (CNN) have the potential to predict polyp pathology and achieve PIVI thresholds in real-time.Methods-We developed a CNN-based optical pathology (OP) model using Tensorflow and pretrained on ImageNet, capable of operating at 77 frames per second. 6223 images of unique colorectal polyps of known pathology, location, size, and light source (white light [WL] or narrow band imaging [NBI]) underwent 5-fold cross training (80%) and validation (20%). Separate fresh validation was performed on 634 polyp images. Surveillance intervals were calculated, comparing OP vs. true pathology (TP).Results-In the original validation set, the negative predictive value (NPV) for adenomas was 97% among diminutive rectum/rectosigmoid polyps. Results were independent of use of NBI or WL. Surveillance interval concordance comparing OP and TP was 93%. In the fresh validation set, NPV was 97% among diminutive polyps in the rectum and rectosigmoid and surveillance concordance was 94%. Conclusion:This study demonstrates the feasibility of in situ diagnosis of colorectal polyps using CNN. Our model exceeds PIVI thresholds for both "resect and discard" and "diagnose and

No abstract

No abstract