A Survey on Graph-Based Deep Learning for Computational Histopathology

Ahmedt-Aristizabal, David; Armin, Mohammad Ali; Denman, Simon; Fookes, Clinton

doi:10.48550/arxiv.2107.00272

Cited by 3 publications

(2 citation statements)

References 104 publications

(219 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recent studies employ graph techniques to embed the biological entities such as cells, tissue regions, and patches into graph-structured data [2,13]. Hence, GNN-based methods become popular to analyze such graph-structured data to facilitate medical decisions.…”

Section: Gnns In Digital Pathologymentioning

confidence: 99%

Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows

Yu¹,

Xu²,

He³

2021

Preprint

View full text Add to dashboard Cite

As Graph Neural Networks (GNNs) are widely adopted in digital pathology, there is increasing attention to developing explanation models (explainers) of GNNs for improved transparency in clinical decisions. Existing explainers discover an explanatory subgraph relevant to the prediction. However, such a subgraph is insufficient to reveal all the critical biological substructures for the prediction because the prediction will remain unchanged after removing that subgraph. Hence, an explanatory subgraph should be not only necessary for prediction, but also sufficient to uncover the most predictive regions for the explanation. Such explanation requires a measurement of information transferred from different input subgraphs to the predictive output, which we define as information flow. In this work, we address these key challenges and propose IFEXPLAINER, which generates a necessary and sufficient explanation for GNNs. To evaluate the information flow within GNN's prediction, we first propose a novel notion of predictiveness, named f -information, which is directional and incorporates the realistic capacity of the GNN model. Based on it, IFEXPLAINER generates the explanatory subgraph with maximal information flow to the prediction. Meanwhile, it minimizes the information flow from the input to the predictive result after removing the explanation. Thus, the produced explanation is necessarily important to the prediction and sufficient to reveal the most crucial substructures. We evaluate IFEXPLAINER to interpret GNN's predictions on breast cancer subtyping. Experimental results on the BRACS dataset show the superior performance of the proposed method.

show abstract

Section: Gnns In Digital Pathologymentioning

confidence: 99%

Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows

Yu¹,

Xu²,

He³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Interestingly, when the graphnodes depict biological entities, e.g., nuclei, tissue regions, the entity-graphs combined with feature attribution techniques can provide pathologist-friendly interpretations (Zhou et al, 2019;Jaume et al, 2020;Sureka et al, 2020) and explanations (Jaume et al, 2021), unlike pixelated blurry saliency maps. A detailed review of graphs in computational pathology is presented by Ahmedt-Aristizabal et al (2021).…”

Section: Graphs In Computational Pathologymentioning

confidence: 99%

HistoCartography: A Toolkit for Graph Analytics in Digital Pathology

Jaume¹,

Pati²,

Anklin³

et al. 2021

Preprint

View full text Add to dashboard Cite

Advances in entity-graph based analysis of histopathology images have brought in a new paradigm to describe tissue composition, and learn the tissue structure-to-function relationship. Entity-graphs offer flexible and scalable representations to characterize tissue organization, while allowing the incorporation of prior pathological knowledge to further support model interpretability and explainability. However, entity-graph analysis requires prerequisites for image-to-graph translation and knowledge of state-of-the-art machine learning algorithms applied to graph-structured data, which can potentially hinder their adoption. In this work, we aim to alleviate these issues by developing HistoCartography, a standardized python API with necessary preprocessing, machine learning and explainability tools to facilitate graph-analytics in computational pathology. Further, we have benchmarked the computational time and performance on multiple datasets across different imaging types and histopathology tasks to highlight the applicability of the API for building computational pathology workflows.

show abstract

Computational staining of CD3/CD20 positive lymphocytes in human tissues with experimental confirmation in a genetically engineered mouse model

Li,

Heirman,

Rickard

et al. 2024

Front. Immunol.

View full text Add to dashboard Cite

IntroductionImmune dysregulation plays a major role in cancer progression. The quantification of lymphocytic spatial inflammation may enable spatial system biology, improve understanding of therapeutic resistance, and contribute to prognostic imaging biomarkers.MethodsIn this paper, we propose a knowledge-guided deep learning framework to measure the lymphocytic spatial architecture on human H&E tissue, where the fidelity of training labels is maximized through single-cell resolution image registration of H&E to IHC. We demonstrate that such an approach enables pixel-perfect ground-truth labeling of lymphocytes on H&E as measured by IHC. We then experimentally validate our technique in a genetically engineered, immune-compromised Rag2 mouse model, where Rag2 knockout mice lacking mature lymphocytes are used as a negative experimental control. Such experimental validation moves beyond the classical statistical testing of deep learning models and demonstrates feasibility of more rigorous validation strategies that integrate computational science and basic science.ResultsUsing our developed approach, we automatically annotated more than 111,000 human nuclei (45,611 CD3/CD20 positive lymphocytes) on H&E images to develop our model, which achieved an AUC of 0.78 and 0.71 on internal hold-out testing data and external testing on an independent dataset, respectively. As a measure of the global spatial architecture of the lymphocytic microenvironment, the average structural similarity between predicted lymphocytic density maps and ground truth lymphocytic density maps was 0.86 ± 0.06 on testing data. On experimental mouse model validation, we measured a lymphocytic density of 96.5 ± %1% in a Rag2+/- control mouse, compared to an average of 16.2 ± %5% in Rag2-/- immune knockout mice (p<0.0001, ANOVA-test).DiscussionThese results demonstrate that CD3/CD20 positive lymphocytes can be accurately detected and characterized on H&E by deep learning and generalized across species. Collectively, these data suggest that our understanding of complex biological systems may benefit from computationally-derived spatial analysis, as well as integration of computational science and basic science.

show abstract

A Survey on Graph-Based Deep Learning for Computational Histopathology

Cited by 3 publications

References 104 publications

Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows

Towards the Explanation of Graph Neural Networks in Digital Pathology with Information Flows

HistoCartography: A Toolkit for Graph Analytics in Digital Pathology

Computational staining of CD3/CD20 positive lymphocytes in human tissues with experimental confirmation in a genetically engineered mouse model

Contact Info

Product

Resources

About