Knowledge-Infused Global-Local Data Fusion for Spatial Predictive Modeling in Precision Medicine

Glioblastoma states are defined by cohabitating cellular populations with progression-, imaging- and sex-distinct patterns

“…Early radiomics efforts focused on connecting an entire image to a biological prediction, thus failing to capture the spatial heterogeneity within individual tumors. More recently, our group and others have leveraged image-localized biopsies to directly connect spatially-resolved imaging features with tissue biology (42)(43)(44)(45)(46)(47)(48). The differences in bioimaging relationships seen in the primary and recurrent setting as well as between the sexes embolden the careful incorporation of these variables in the context of radiomics.…”

Biologically-informed deep neural networks provide quantitative assessment of intratumoral heterogeneity in post-treatment glioblastoma

“…Some researchers proposed to use the knowledge of biological pathways to guide the architecture design of DL (31, 32). Moreover, in some biomedical fields, domain knowledge exists in the form of algebraic equations representing biological principles which are imposed on ML models (24, 34). Additionally, some researchers proposed to integrate the knowledge about how features should behave in generating prediction as attribution priors into DL training (33).…”

“…Second, integration of biological/biomedical domain knowledge, such as, biological principles, empirical models, simulations, and knowledge graphs, can provide a rich source of information (pseudo data) to help alleviate the data shortage in training DL models. Biologically-or biomedically-informed deep learning (BIDL) has been proposed to use domain knowledge to guide the design of DL architecture (31,32), as attribution priors of features (33), to regularize the model predictions, coefficients, or latent feature representations (24,34), and implicitly leveraging knowledge from other domains through transfer learning (35)(36)(37)(38). However, these existing works lack the capability of integrating hierarchical domain knowledge which is hard to be described in mathematical formulation.…”

“…Moreover, the lack of large datasets has severely limited the number of studies focusing on predicting regional characteristics within each lesion, which are crucial for revealing intratumoral heterogeneity. A few studies have developed MRI-biology fusion models to predict regional cell density (21)(22)(23)(24)(25)(26) or regional copy number variation of individual driver genes such as EGFR, PDGFRA, and PTEN, in untreated, primary GBM (27,28). In recurrent GBM (recGBM), however, treatment-induced reactive changes lead to additional intratumoral heterogeneity and the related additional complexity in tissue composition makes prediction of gene modules more difficult (29).…”

“…For example, some researchers proposed to use the knowledge of biological pathways to guide the design of DL architecture (31, 32). In certain biomedical domains, knowledge exists in the form of algebraic equations that capture biological principles, which were integrated with DL architecture or loss functions (23, 33). Some researchers proposed to integrate knowledge about feature behavior as attribution priors into DL training (34).…”