Quality control stress test for deep learning-based diagnostic model in digital pathology

Abstract

Digital pathology provides a possibility for computational analysis of histological slides and automatization of routine pathological tasks. Histological slides are very heterogeneous concerning staining, sections’ thickness, and artifacts arising during tissue processing, cutting, staining, and digitization. In this study, we digitally reproduce major types of artifacts. Using six datasets from four different institutions digitized by different scanner systems, we systematically explore artifacts’ influence on the accuracy of the pre-trained, validated, deep learning-based model for prostate cancer detection in histological slides. We provide evidence that any histological artifact dependent on severity can lead to a substantial loss in model performance. Strategies for the prevention of diagnostic model accuracy losses in the context of artifacts are warranted. Stress-testing of diagnostic models using synthetically generated artifacts might be an essential step during clinical validation of deep learning-based algorithms.

Artificial intelligence for detection of microsatellite instability in colorectal cancer—a multicentric analysis of a pre-screening tool for clinical application

“…These slides should have been—but were not—excluded before the computational image analysis which shows the importance of rigorous quality control. 37 For another seven (32%) of the misclassified slides, no reason for misclassification could be identified. In addition, potential reasons for FP misclassification were analyzed ( Figure 4 B), although the level of concern about FP cases is low for a pre-screening test.…”

Artificial Intelligence-based Tumor Segmentation in Mouse Models of Lung Adenocarcinoma

“…It is well documented that variation in staining conditions or tissue processing artifacts can negatively impact the performance of deep learning models. 37 , 38 Related, despite controlling for different experimental cohorts of mice, we did not have an external cohort to evaluate the generalizability of these models. Finally, a large number of small false-positive regions may indicate the model will require future fine-tuning for users who wish to capture tumor foci characterized by few-to-several individual cells.…”

Recommendations on test datasets for evaluating AI solutions in pathology

“…Differences in section thickness and staining solutions can lead to variable staining appearances [39]. Artifacts frequently occur during tissue processing, including elastic deformations, inclusion of foreign objects, and cover glass scratches [38]. Di erences in illumination, resolution, and encoding algorithms of slide scanner models also a ect the appearance of tissue images [36].…”

“…There are a variety of techniques for extending datasets with synthetic data. Some techniques alter existing images in a generic (e.g., rotation, mirroring) or histologyspeci c way (e.g., stain transformations [26] or emulation of image artifacts [38,[76][77][78][79][80][81]). Other techniques create fully synthetic images from scratch [82][83][84][85][86].…”