Deep Learning for Classification of Colorectal Polyps on Whole-slide Images

Abstract

Context:Histopathological characterization of colorectal polyps is critical for determining the risk of colorectal cancer and future rates of surveillance for patients. However, this characterization is a challenging task and suffers from significant inter- and intra-observer variability.Aims:We built an automatic image analysis method that can accurately classify different types of colorectal polyps on whole-slide images to help pathologists with this characterization and diagnosis.Setting and Design:Our method is based on deep-learning techniques, which rely on numerous levels of abstraction for data representation and have shown state-of-the-art results for various image analysis tasks.Subjects and Methods:Our method covers five common types of polyps (i.e., hyperplastic, sessile serrated, traditional serrated, tubular, and tubulovillous/villous) that are included in the US Multisociety Task Force guidelines for colorectal cancer risk assessment and surveillance. We developed multiple deep-learning approaches by leveraging a dataset of 2074 crop images, which were annotated by multiple domain expert pathologists as reference standards.Statistical Analysis:We evaluated our method on an independent test set of 239 whole-slide images and measured standard machine-learning evaluation metrics of accuracy, precision, recall, and F1 score and their 95% confidence intervals.Results:Our evaluation shows that our method with residual network architecture achieves the best performance for classification of colorectal polyps on whole-slide images (overall accuracy: 93.0%, 95% confidence interval: 89.0%–95.9%).Conclusions:Our method can reduce the cognitive burden on pathologists and improve their efficacy in histopathological characterization of colorectal polyps and in subsequent risk assessment and follow-up recommendations.

Evaluation of a Deep Neural Network for Automated Classification of Colorectal Polyps on Histopathologic Slides

“…Our study not only demonstrates the utility of a deep learning model for classification of colorectal polyps but also advances previous literature in terms of model evaluation and study design. The previous foremost study on deep learning for colorectal polyp classification, done by our team, 26,27 demonstrated good performance on an internal dataset but used a simpler approach and did not include pathologist-level performance or local diagnoses. Our study, on the other hand, evaluates a deep neural network on a multi-institutional external dataset and demonstrates a comparable diagnostic performance of deep neural networks compared to local pathologists at the point-of-care.…”

Dealing with Lack of Training Data for Convolutional Neural Networks: The Case of Digital Pathology

“…Among feature learning methods, deep learning and, more specifically, Convolutional Neural Networks (CNNs) have now become a major trend in many computer vision and medical tasks [10][11][12]. In CNNs, a number of convolutional and pooling layers learn by backpropagation the set of features that are best for classification, thus avoiding the design of handcrafted texture descriptors.…”

Diagnostic Performance of Deep Learning Algorithms Applied to Three Common Diagnoses in Dermatopathology

“…Artificial intelligence, and specifically deep learning, is now being applied to the fields of clinical dermatology and pathology. [ 39 40 41 42 43 44 45 46 47 48 ] Both specialties are image intense, rely on the integration of visual skills for diagnosis, and are a natural segue for algorithmic development. Esteva et al .…”