Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples

Abstract

Background:Immune cell infiltration in tumor is an emerging prognostic biomarker in breast cancer. The gold standard for quantification of immune cells in tissue sections is visual assessment through a microscope, which is subjective and semi-quantitative. In this study, we propose and evaluate an approach based on antibody-guided annotation and deep learning to quantify immune cell-rich areas in hematoxylin and eosin (H&E) stained samples.Methods:Consecutive sections of formalin-fixed parafin-embedded samples obtained from the primary tumor of twenty breast cancer patients were cut and stained with H&E and the pan-leukocyte CD45 antibody. The stained slides were digitally scanned, and a training set of immune cell-rich and cell-poor tissue regions was annotated in H&E whole-slide images using the CD45-expression as a guide. In analysis, the images were divided into small homogenous regions, superpixels, from which features were extracted using a pretrained convolutional neural network (CNN) and classified with a support of vector machine. The CNN approach was compared to texture-based classification and to visual assessments performed by two pathologists.Results:In a set of 123,442 labeled superpixels, the CNN approach achieved an F-score of 0.94 (range: 0.92–0.94) in discrimination of immune cell-rich and cell-poor regions, as compared to an F-score of 0.88 (range: 0.87–0.89) obtained with the texture-based classification. When compared to visual assessment of 200 images, an agreement of 90% (κ = 0.79) to quantify immune infiltration with the CNN approach was achieved while the inter-observer agreement between pathologists was 90% (κ = 0.78).Conclusions:Our findings indicate that deep learning can be applied to quantify immune cell infiltration in breast cancer samples using a basic morphology staining only. A good discrimination of immune cell-rich areas was achieved, well in concordance with both leukocyte antigen expression and pathologists’ visual assessment.

Machine Learning Methods for Histopathological Image Analysis

“…Tasks related to this analysis include detection of immune cells from H&E stained image [121,122] and detection of more specific type of immune cells using immunohistochemistry [104]. Additionally, the pattern of immune cell infiltration and proximity of each immune cells are reportedly related to cancer prognosis [123], analysis of spatial relationships between tumor cells and immune cells, and the relationships between these data and prognosis or response to immunotherapy using specialized algorithms such as graph-based algorithms [63,124] will also be of great importance.…”

Emerging role of deep learning‐based artificial intelligence in tumor pathology

“…It has been reported that the kind and amount of tumor‐infiltrating immune cells are related with the sensitivity to immunotherapy and prognostic stratification for tumor patients [60, 61]. A DL method using CD45‐annotated digital images could quantify immune cells and distinguish immune cell‐rich or ‐poor regions in breast cancer [62]. The AUC was 0.99 without the limitation of histological types and grades.…”

Implementation of deep neural networks to count dopamine neurons in substantia nigra

“…Convolutional neural networks (CNN) are ANNs that are especially powerful for pattern recognition in digital images (Fukushima, ; Lecun, Bottou, Bengio, & Haffner, ). In recent years, the technological advances in graphical processing units have enabled efficient implementation of deep CNNs in biological image analysis with highly promising results (Ciresan, Giusti, Gambardella, & Schmidhuber, ; Kraus et al., ; LeCun, Bengio, & Hinton, ; Mamoshina, Vieira, Putin, & Zhavoronkov, ; Turkki, Linder, Kovanen, Pellinen, & Lundin, ). Thus, machine learning provides a great possibility to enhance the throughput of analysis of biological samples and to produce new knowledge.…”