Multi-Cell Type and Multi-Level Graph Aggregation Network for Cancer Grading in Pathology Images

Bi-level Graph Learning Unveils Prognosis-Relevant Tumor Microenvironment Patterns in Breast Multiplexed Digital Pathology

“…Modern deep learning methods, including graph neural networks, constitute an appealing alternative for data-driven biomarkers for prognosis [21, 22]. Yet, their employment in real-world clinical settings remains constrained by the lack of rigorous external validation [35, 36] and limited explain-ability [25, 26, 28, 29, 32, 33]. Instead, this study introduces a data-driven methodology centered on breast cancer, that relies on two simple observations: i) representative phenotypic patterns of the TME can be adaptively learned by studying the underlying patient-specific cellular graph constructed from multiplexed tissue image data, and ii) the relative abundance of such patterns in different patients can be employed to provide a measure of their similarity, from which a population-level graph can be constructed.…”

“…On the other hand, there is a growing interest in harnessing graph-based learning techniques to analyze the association between the TME and disease without the need for explicit hypotheses in a data-driven manner [21,22]. Numerous recent studies have reported promising results by leveraging graph neural networks (GNNs) to model the TME and predict the presence [23,24], grade [25,26], stage [27], subtype [28,29], and prognosis [30][31][32][33][34] of diverse types of cancers. Yet, the limited size, biological heterogeneity, and differences in staining and imaging protocols of clinical datasets constrain the quality of modern machine learning models by impeding their generalization capacity, particularly in crossstudy scenarios that are often overlooked [35,36].…”

“…Additionally, the interpretability of deep learning models, in general, remains a challenge [37, 38]. Many of these studies do not discuss explanation in their works [25, 26, 28, 29, 32, 33], some employ gradient-based [23, 27, 34] and permutation-based [24] methods to generate an importance heatmap as the models’ explanation, and others conduct post-hoc analyses on learned graph features to obtain some clinical implications from their models [30, 31]. These explanations, although providing some insights into learned information, might not be clear, intuitive, or meaningful, especially for clinical experts and cancer researchers.…”

Computational pathology: A survey review and the way forward

“… 268 Some works explicitly encode the patient-slide-patch hierarchy in the attention mechanism, 457 , 458 with one work using a cellular graph for top-down attention. 459 Graph Neural Networks (GNNs) have been explored to leverage intra- and inter-cell relationships, enabling cancer grading, 460 classification, 461 and survival prediction. 462 , 463 These hierarchy- and morphology-aware models are the current SOTA and pave the way for future improvements.…”

“…However, this ignores the spatial relationships between cells and tissues or between the patches and their parent slides in histopathology images, which are often relevant or even crucial when making decisions. Many works have recently found success in explicitly encoding an awareness of these inter-cell relationships 460 , 462 , 463 and the patch-slide-patient hierarchy, 457 , 458 , 459 especially using Graph neural networks, but these suffer from higher latency than conventional CNNs. We anticipate future works will seek to speed up GNNs for tasks where spatial and hierarchical relationships are important and continue developing hierarchy-aware attention for MIL techniques.…”