Deep collective matrix factorization for augmented multi-view learning

Abstract

Learning by integrating multiple heterogeneous data sources is a common requirement in many tasks. Collective Matrix Factorization (CMF) is a technique to learn shared latent representations from arbitrary collections of matrices. It can be used to simultaneously complete one or more matrices, for predicting the unknown entries. Classical CMF methods assume linearity in the interaction of latent factors which can be restrictive and fails to capture complex non-linear interactions. In this paper, we develop the first deep-learning based method, called dCMF, for unsupervised learning of multiple shared representations, that can model such non-linear interactions, from an arbitrary collection of matrices. We address optimization challenges that arise due to dependencies between shared representations through Multi-Task Bayesian Optimization and design an acquisition function adapted for collective learning of hyperparameters. Our experiments show that dCMF significantly outperforms previous CMF algorithms in integrating heterogeneous data for predictive modeling. Further, on two tasks -recommendation and prediction of gene-disease association -dCMF outperforms state-of-the-art matrix completion algorithms that can utilize auxilliary sources of information.

Multi-way Clustering and Discordance Analysis through Deep Collective Matrix Tri-Factorization

“…Representation learning from arbitrary collections of matrices have been studied in Collective Matrix Factorization (CMF) [Singh and Gordon, 2008], group-sparse CMF [Klami et al, 2014] and a neural approach Deep CMF [Mariappan and Rajan, 2019]. These approaches learn two latent factors for the row and column entities of each matrix, to reconstruct them.…”

“…Entity-specific representations may be learned by using N different autoencoders, one for each entity, where each autoencoder takes as input the concatenation of all matrices containing that entity. This approach is inadequate for matrices with different datatypes and sparsity levels as discussed in [Mariappan and Rajan, 2019]. Our approach addresses these problems but with higher computational cost.…”

Patient Representation Learning From Heterogeneous Data Sources and Knowledge Graphs Using Deep Collective Matrix Factorization: Evaluation Study

“…In this paper, we analyze patient representation learning in light of 2 recent advances in CMF and KG representation learning. A deep autoencoder-based architecture, called deep CMF (DCMF), was developed for CMF, which was found to outperform classical nonneural variants of CMF in several tasks [ 9 ]. Using DCMF, which provides a seamless way of integrating heterogeneous data, we evaluate the effectiveness of patient representations when the input data are augmented with additional information from literature-derived KGs.…”

“…A model for CMF based on deep learning was developed by Mariappan and Rajan [ 9 ], which is briefly described next. Given M matrices (indexed by m) that describe the relationships between E entities (indexed by e), each with dimension d e, DCMF jointly obtains latent representations of each entity U e and low-rank factorizations of each matrix such that U e =f θ ([C] (e) ), where f θ is an entity-specific nonlinear transformation, obtained through a neural network–based encoder with weights θ and [C] (e) denotes all matrices in the collection that contain a relationship of entity e. The entities corresponding to the rows and columns of the m th matrix are denoted by indices r m and c m , respectively.…”

“…This makes collective learning of all latent representations challenging and, to scale to arbitrary collections of matrices, necessitates automatic hyperparameter selection. We address these optimization challenges through multitask Bayesian optimization (details can be found in the study by Mariappan and Rajan [ 9 ]).…”