GraphSAIL

Abstract

Given the convenience of collecting information through online services, recommender systems now consume large scale data and play a more important role in improving user experience. With the recent emergence of Graph Neural Networks (GNNs), GNN-based recommender models have shown the advantage of modeling the recommender system as a user-item bipartite graph to learn representations of users and items. However, such models are expensive to train and difficult to perform frequent updates to provide the most up-to-date recommendations. In this work, we propose to update GNN-based recommender models incrementally so that the computation time can be greatly reduced and models can be updated more frequently. We develop a Graph Structure Aware Incremental Learning framework, GraphSAIL, to address the commonly experienced catastrophic forgetting problem that occurs when training a model in an incremental fashion. Our approach preserves a user's long-term preference (or an item's long-term property) during incremental model updating. GraphSAIL implements a graph structure preservation strategy which explicitly preserves each node's local structure, global structure, and self-information, respectively. We argue that our incremental training framework is the first attempt tailored for GNN based recommender systems and demonstrate its improvement compared to other incremental learning techniques on two public datasets. We further verify the effectiveness of our framework on a large-scale industrial dataset. CCS CONCEPTS • Information systems → Recommender systems; • Computing methodologies → Online learning settings.

KHGCN: Knowledge-Enhanced Recommendation with Hierarchical Graph Capsule Network

“…In the online recommendation services, Xu et al. developed a framework based on incremental learning graph neural networks to address the catastrophic forgetting problem faced during incremental learning training, which implements a graph structure preservation strategy to preserve users’ long-term preferences when the model is updated [ 49 ]. In the personalized video highlight recommendations, Wu et al.…”

A Survey on Incremental Update for Neural Recommender Systems

“…Through treating past models as the teacher model, Wang et al [94] regularize the learning process with a knowledge distillation loss to prevent catastrophic forgetting. Additionally, subsequent studies employ distillation techniques with specific attention to GNNs [99] and session-based RSs [66]. However, it is important to note that simply limiting the change in networks will not lead to transferring useful knowledge to current tasks.…”

“…In other words, some of knowledge in past models might not be applicable for the current task. Zhang et al [109] introduce a method for explicitly optimizing the next period's data, which greatly alleviates the problem of [66,94,99]. To optimize for future use, it develops a transfer module to combine knowledge of different periods' models.…”

Dynamically Expandable Graph Convolution for Streaming Recommendation

“…Due to the strong capability of modeling user-item, user-user, and item-item relationships, graph convolution neural network (GCN) has been widely-used as recommendation models. Some recent works [1,51,52,56,61] develop continual learning methods for GCN-based recommendation methods to achieve the streaming recommendation, also known as continual graph learning for streaming recommendation.…”

“…To enable continual GCN-based recommendation, most works focus on two realizations: experience replay [1,52,68] and knowledge distillation/weight regularization [29,51,56,61]. Although these methods have achieved acceptable results, there are still drawbacks which hinder them from being applied in the real-world systems.…”