Variational Autoencoders for Collaborative Filtering

Abstract

We extend variational autoencoders (vaes) to collaborative filtering for implicit feedback. This non-linear probabilistic model enables us to go beyond the limited modeling capacity of linear factor models which still largely dominate collaborative filtering research. We introduce a generative model with multinomial likelihood and use Bayesian inference for parameter estimation. Despite widespread use in language modeling and economics, the multinomial likelihood receives less attention in the recommender systems literature. We introduce a different regularization parameter for the learning objective, which proves to be crucial for achieving competitive performance. Remarkably, there is an efficient way to tune the parameter using annealing. The resulting model and learning algorithm has information-theoretic connections to maximum entropy discrimination and the information bottleneck principle. Empirically, we show that the proposed approach significantly outperforms several state-of-the-art baselines, including two recently-proposed neural network approaches, on several real-world datasets. We also provide extended experiments comparing the multinomial likelihood with other commonly used likelihood functions in the latent factor collaborative filtering literature and show favorable results. Finally, we identify the pros and cons of employing a principled Bayesian inference approach and characterize settings where it provides the most significant improvements.

RecVAE: A New Variational Autoencoder for Top-N Recommendations with Implicit Feedback

“…The standard solution is to use simple L 2 or L 1 regularizers for the embedding weights, although more flexible priors have also been used in literature [20,29]. The works [22,39] present an alternative way of regularization based on the amortization of user embeddings coupled with denoising [16,31,37]. Several more models are reviewed in Section 4.3.…”

“…We begin with a description of the Mult-VAE model proposed in [22]. The basic idea of Mult-VAE is similar to VAE but with the multinomial distribution as the likelihood function instead of Gaussian and Bernoulli distributions commonly used in VAE.…”

“…Note that while the original paper [22] compares Mult-VAE with Mult-DAE, a denoising autoencoder that applies Bernoulli-based noise to the input but does not have the VAE structure (Mult-DAE is a regular denoising autoencoder), in reality the authors used denoising for Mult-VAE as well. This is evidenced both by their code base and by our experiments, where we were able to match the results of [22] when we used denoising and got nowhere even close without denoising (we will return to this discussion in Section 3.5).…”

“…Note that while the original paper [22] compares Mult-VAE with Mult-DAE, a denoising autoencoder that applies Bernoulli-based noise to the input but does not have the VAE structure (Mult-DAE is a regular denoising autoencoder), in reality the authors used denoising for Mult-VAE as well. This is evidenced both by their code base and by our experiments, where we were able to match the results of [22] when we used denoising and got nowhere even close without denoising (we will return to this discussion in Section 3.5). According to our intuition and experiments, the input noise for the denoising autoencoder and latent variable noise of Monte Carlo integration play different roles: the former forces the model not only to reconstruct the input vector but also to predict unobserved feedback, while the latter leads to more robust embedding learning.…”

“…We have evaluated RecVAE on the MovieLens-20M dataset 1 [9], Netflix Prize Dataset 2 [3], and Million Songs Dataset 3 [4]. We have preprocessed the datasets in accordance with the Mult-VAE approach [22]. Dataset statistics after preprocessing are as follows: MovieLens-20M contains 9,990,682 ratings on 20,720 movies provided by 136,677 users, Netflix Prize Dataset contains 56,880,037 ratings on 17,769 movies provided by 463,435 users, and Million Songs Dataset contains 33,633,450 ratings on 41,140 songs provided by 571,355 users.…”