Relational Learning via Collective Matrix Factorization

Abstract

Relational learning is concerned with predicting unknown values of a relation, given a database of entities and observed relations among entities. An example of relational learning is movie rating prediction, where entities could include users, movies, genres, and actors. Relations would then encode users' ratings of movies, movies' genres, and actors' roles in movies. A common prediction technique given one pairwise relation, for example a #users × #movies ratings matrix, is low-rank matrix factorization. In domains with multiple relations, represented as multiple matrices, we may improve predictive accuracy by exploiting information from one relation while predicting another. To this end, we propose a collective matrix factorization model: we simultaneously factor several matrices, sharing parameters among factors when an entity participates in multiple relations. Each relation can have a different value type and error distribution; so, we allow nonlinear relationships between the parameters and outputs, using Bregman divergences to measure error. We extend standard alternating projection algorithms to our model, and derive an efficient Newton update for the projection. Furthermore, we propose stochastic optimization methods to deal with large, sparse matrices. Our model generalizes several existing matrix factorization methods, and therefore yields new large-scale optimization algorithms for these problems. Our model can handle any pairwise relational schema and a wide variety of error models. We demonstrate its efficiency, as well as the benefit of sharing parameters among relations.

Learning Preferences with Side Information

“…In contrast, our approach centers on interactions between users and items, but we will see that it is flexible enough to incorporate user and item features as well. Another approach is collective matrix factorization (Singh and Gordon (2008)), which studies matrix recovery using multiple matrices across multiple groups. In the setting of multiple matrices across two groups, our approach will in fact apply to a more general version of this problem and simultaneously allow us to leverage the modeling advantages of tensors.…”

Discriminative Factor Alignment across Heterogeneous Feature Space

“…Furthermore, a comparison is made between HTLFA and CMF(i.e. Collective Matrix Factorization) [21]. In CMF, they simultaneously factor several matrices, sharing parameters among factors when an entity participates in multiple relations.…”

Non‐negative residual matrix factorization: problem definition, fast solutions, and applications

“…This means that we only measure the element‐wise loss on the observed edges; and among all these edges, we treat the element‐wise loss equally (referred to as ‘0/1 Weight Matix’). This type of weight matrix in widely used in the literature, especially in the context of collaborative filtering [6,7].…”