A knowledge representation framework for integration, classification of multi-scale imaging and non-imaging data: Preliminary results in predicting prostate cancer recurrence by fusing mass spectrometry and histology

Abstract

The demand for personalized health care requires a wide range of diagnostic tools for determining patient prognosis and theragnosis (response to treatment). These tools present us with data that is both multi-modal (imaging and non-imaging) and multi-scale (proteomics, histology). By utilizing the information in these sources concurrently, we expect significant improvement in predicting patient prognosis and theragnosis. However, a prerequisite to realizing this improvement is the ability to effectively and quantitatively combine information from disparate sources. In this paper, we present a general fusion framework (GFF) aimed towards a combined knowledge representation predicting disease recurrence. To the best of our knowledge, GFF represents the first formal attempt to fuse biomedical image and non-image information directly at the data level as opposed to the decision level, thus preserving more subtle contributions in the original data. GFF represents the different data streams in separate embedding spaces via the application of dimensionality reduction (DR). Data fusion is then implemented by combining the individual reduced embedding spaces. A proof of concept example is considered for evaluating the GFF, whereby protein expression measurements from mass spectrometry are combined with histological image signatures to predict prostate cancer (CaP) recurrence in 6 CaP patients, following therapy. Preliminary results suggest that GFF offers an intelligent way to fuse image and non-image data structures for making prognostic and theragnostic predictions.

Enhanced multi-protocol analysis via intelligent supervised embedding (EMPrAvISE): detecting prostate cancer on multi-parametric MRI

“…A COI approach has typically been shown to be sub-optimal as inter-protocol dependencies are not accounted for. 1 Thus, a number of COD strategies with the express purpose of building integrated quantitative meta-classifiers have recently been presented, including DR-based, 1 kernel-based 11 and feature-based 12 approaches.…”

“…Lee et al 1 proposed data representation and subsequent fusion of the different modalities in a “meta-space” constructed using DR methods such as Graph Embedding 7 (GE). However, DR analysis of a high-dimensional feature space may not necessarily yield optimal results for multi-parametric representation and fusion due to (a) noise in the original N -D space which may adversely affect the embedding projection, or (b) sensitivity to choice of parameters being specified during DR. For example, GE is known to suffer from issues relating to the scale of analysis as well as to the choice of parameters used in the method.…”

“…1, 2 Such multi-channel meta-classifiers have been shown to perform significantly better compared to any individual data channel. 1 From an intuitive perspective, this is because the different channels of information each capture complementary sets of information.…”

“…1, 2 Such multi-channel meta-classifiers have been shown to perform significantly better compared to any individual data channel. 1 From an intuitive perspective, this is because the different channels of information each capture complementary sets of information. For example, the detection accuracy and qualitative characterization of prostate cancer (CaP) in vivo has been shown to significantly improve when multiple magnetic resonance imaging (MRI) protocols are considered in combination, as compared to using individual imaging protocols.…”

Feature Selection Study on Separate Multi-modal Datasets: Application on Cutaneous Melanoma

“…On the other hand, Jesneck et al [4], on a COI path, optimized a decision-fusion technique to combine heterogeneous breast cancer data. Lee et al [5], proposed a Generalized Fusion Framework (GFF) for homogenous data representation and subsequent fusion in the meta-space using dimensionality reduction techniques. The meta-space is created by projecting the heterogeneous data streams into a space where these scale and dimensionality differences are alleviated.…”