Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems

Abstract

Recommender systems that can learn from cross-session data to dynamically predict the next item a user will choose are crucial for online platforms. However, existing approaches often use out-ofthe-box sequence models which are limited by speed and memory consumption, are often infeasible for production environments, and usually do not incorporate cross-session information, which is crucial for effective recommendations. Here we propose Hierarchical Temporal Convolutional Networks (HierTCN), a hierarchical deep learning architecture that makes dynamic recommendations based on users' sequential multi-session interactions with items. HierTCN is designed for web-scale systems with billions of items and hundreds of millions of users. It consists of two levels of models: The high-level model uses Recurrent Neural Networks (RNN) to aggregate users' evolving long-term interests across different sessions, while the low-level model is implemented with Temporal Convolutional Networks (TCN), utilizing both the long-term interests and the short-term interactions within sessions to predict the next interaction. We conduct extensive experiments on a public XING dataset and a large-scale Pinterest dataset that contains 6 million users with 1.6 billion interactions. We show that HierTCN is 2.5x faster than RNN-based models and uses 90% less data memory compared to TCN-based models. We further develop an effective data caching scheme and a queue-based mini-batch generator, enabling our model to be trained within 24 hours on a single GPU. Our model consistently outperforms state-of-the-art dynamic recommendation methods, with up to 18% improvement in recall and 10% in mean reciprocal rank.

Déjà vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation

“…Other methods based on Convolutional Neural Networks (CNN) [40], Memory Network [5] and Attention Models [22] have also been explored. The hierarchical structure generalized from RNN, Attention or CNN based models [7,31,45] is used to model transitions inter-and intra-sessions. The recent work [45] by You et al showed that using Temporal Convolutional Network to encode and decode session-level information and GRU for user-level transition is the most effective hierarchical structure.…”

“…The hierarchical structure generalized from RNN, Attention or CNN based models [7,31,45] is used to model transitions inter-and intra-sessions. The recent work [45] by You et al showed that using Temporal Convolutional Network to encode and decode session-level information and GRU for user-level transition is the most effective hierarchical structure. Nevertheless, as many studies borrow sequence models from natural language modeling task directly, their model performance is usually limited by the relatively small size and sparse pattern of user behaviors, compared to the nature language datasets.…”

“…Each action is associated with the user ID, item ID, action timestamp and interaction type (click, bookmark, delete, etc.). Following the prior work [31,45], we removed interactions with type "delete" and did not consider the interaction types in the data. We removed items associated with less than 50 actions, and removed users with less than 10 or more than 1000 actions.…”

“…Concerning such sequential dependence within user preferences, the task of sequential recommendation is set to predict the ongoing relevant items based on a sequence of the user's historical actions. Such setting has been widely studied [4,5,7,16,19,31,40,45] and practiced in popular industry recommender systems such as YouTube [3,39] and Taobao [38]. Take the online shopping scenario illustrated in Figure 1 for example: the system is given a series of user behavior records and needs to recommend the next set of items for the user to examine.…”

“…It is shown in [16,31] that the pattern of user preference transition usually differs after a session gap, which is commonly defined as a minimal of 30 minutes' inactivity. Prior work has demonstrated many effective hierarchical neural network structures, such as hierarchical recurrent neural networks (RNN) [7,31,45] to jointly model the transition patterns in-and cross-sessions. However, the strength stemmed from its session assumption could also be the bottleneck of session based recommendation, that is, the user preference does not necessarily transit in strict accordance with the manually defined session boundaries.…”