A deep learning-based model of normal histology

Abstract

Deep learning models have been applied on various tissues in order to recognize malignancies. However, these models focus on relatively narrow tissue context or well-defined pathologies. Here, instead of focusing on pathologies, we introduce models characterizing the diversity of normal tissues. We obtained 1,690 slides with rat tissue samples from the control groups of six preclinical toxicology studies, on which tissue regions were outlined and annotated by pathologists into 46 different tissue classes. From these annotated regions, we sampled small patches of 224 x 224 pixels at six different levels of magnification. Using four studies as training set and two studies as test set, we trained VGG-16, ResNet-50, and Inception-v3 networks separately at each of these magnification levels. Among these models, Inception-v3 consistently outperformed the other networks and attained accuracies up to 83.4% (top-3 accuracy: 96.3%). Further analysis showed that most tissue confusions occurred within clusters of histologically similar tissues. Investigation of the embedding layer using the UMAP method revealed not only pronounced clusters corresponding to the individual tissues, but also subclusters corresponding to histologically meaningful structures that had neither been annotated nor trained for. This suggests that the histological representation learned by the normal histology network could also be used to flag abnormal tissue as outliers in the embedding space without a need to explicitly train for specific types of abnormalities. Finally, we found that models trained on rat tissues can be used on non-human primate and minipig tissues with minimal retraining. Significance statementLike many other scientific disciplines, histopathology has been profoundly impacted by recent advances in machine learning with deep neural networks. In this field, most deep learning models reported in the literature are trained on pathologies in specific tissues/contexts. Here, we aim to establish a model of normal tissues as a foundation for future models of histopathology. We build models that are specific to histopathology images and we show that their embeddings are better feature vectors for describing the underlying images than those of off-the shelf CNN models. Therefore, our models could be useful for transfer learning to improve the accuracy of other histopathology models.

Society of Toxicologic Pathology Digital Pathology and Image Analysis Special Interest Group Article*: Opinion on the Application of Artificial Intelligence and Machine Learning to Digital Toxicologic Pathology

“…Figure 4 shows an example of the t-SNE visualization technique. 51 Each point in this t-SNE plot corresponds to a small image patch, with patches taken from rat tissues of different types (eg, liver, kidney, lung, etc). Patches are colored according to their true tissue type.…”

“…Recent DL models have automated classification of WSI of different, normal, non-diseased rat tissues with 94.7% to 98.3% accuracy, thus providing a potential foundation for future models of toxicologic pathology. 51…”

“…Recent DL models have automated classification of WSI of different, normal, non-diseased rat tissues with 94.7% to 98.3% accuracy, thus providing a potential foundation for future models of toxicologic pathology. 51 The benefits of DL algorithms in investigative toxicologic pathology have already started showing great promise. For instance, as opposed to conventional image analysis algorithms, CNNs can precisely detect and segment glomeruli within digitized kidney sections.…”

“…WF indicates white fat; BF, brown fat; HT, heart; SKN, skin; HG, harderian gland; PAN, pancreas; BKG, background; LA, large intestine; LN, lymph node; LI, liver; TO, tongue; LU, lung; TH, thymus; AG, adrenal gland; MG, mammary gland; KD, kidney; TYD, thyroid gland; SP, spleen; ST, stomach; SI, small intestine; SM, skeletal muscle; SG, salivary gland; UB, urinary bladder. Figure provided courtesy of Sing 51.…”

From RNA to histological images: linking the transcriptome with human phenotypes through statistical learning

“…Existing models of histology in human are mostly focused on diseased tissues and identification of malignancies, but efforts have also been done to build models of normal tissue histology: Sing et al [165] benchmarked the performance of different CNN architectures to recognize 46 different tissues from WSIs derived from rat tissue samples, finding that the generated feature vectors cluster together with respect to individual tissues and that morphologically similar tissues tend to have overlapping clusters. They suggest that these representations can be used to perform histological outlier identification, as well as discussing how these models, although not immediately applicable in their current state, could be used as a basis for cross-species histology predictions.…”