Temporal Collaborative Ranking Via Personalized Transformer

Abstract

The collaborative ranking problem has been an important open research question as most recommendation problems can be naturally formulated as ranking problems. While much of collaborative ranking methodology assumes static ranking data, the importance of temporal information to improving ranking performance is increasingly apparent. Recent advances in deep learning, especially the discovery of various attention mechanisms and newer architectures in addition to widely used RNN and CNN in natural language processing, have allowed us to make better use of the temporal ordering of items that each user has engaged with. In particular, the SASRec model, inspired by the popular Transformer model in natural languages processing, has achieved state-of-art results in the temporal collaborative ranking problem and enjoyed more than 10x speed-up when compared to earlier CNN/RNNbased methods. However, SASRec is inherently an un-personalized model and does not include personalized user embeddings. To overcome this limitation, we propose a Personalized Transformer (SSE-PT) model, outperforming SASRec by almost 5% in terms of NDCG@10 on 5 real-world datasets. Furthermore, after examining some random users' engagement history and corresponding attention heat maps used during the inference stage, we find our model is not only more interpretable but also able to focus on recent engagement patterns for each user. Moreover, our SSE-PT model with a slight modification, which we call SSE-PT++, can handle extremely long sequences and outperform SASRec in ranking results with comparable training speed, striking a balance between performance and speed requirements. Code and data are open sourced at https://github. com/wuliwei9278/SSE-PT.

Advances in Collaborative Filtering and Ranking

“…On one hand, we propose a novel way to encode long range graph interactions without require any training using bloom filters as backbone [119]. On the other hand, with the help of a new embedding-layer regularization called Stochastic Shared Embeddings (SSE) [117], we can also introduce personalization for the state-of-the-art sequential recommendation model and achieve much better ranking performance with our personalized model [118], where personalization is crucial for the success of recommender systems unlike most natural language tasks. This new regularization not only helps existing collaborative filtering and collaborative ranking algorithms but also benefits methods in natural language processing in fields like machine translation and sentiment analysis [117].…”

“…However, to the best of our knowledge, until now, they have not been used to encode graphs, nor has this encoding been applied to recommender systems. So it would be interesting to extend our work to other recommender systems settings, such as [118] and [117].…”