A Survey on Incremental Update for Neural Recommender Systems

Zhang, Peiyan; Kim, Sunghun

doi:10.48550/arxiv.2303.02851

Cited by 3 publications

(3 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, not all news articles in current news platforms have topic labels. From a data perspective, manually labeling topics for a large number of news articles that appear every day is not practical [26]. Therefore, we consider using topic prediction as a learning objective for recommendation, rather than using topic information as input.…”

Section: Topic Perceptronmentioning

confidence: 99%

News Recommendation With Word-Related Joint Topic Prediction

Pu,

Zhang,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Section: Topic Perceptronmentioning

confidence: 99%

News Recommendation With Word-Related Joint Topic Prediction

Pu,

Zhang,

Chen

et al. 2024

IEEE Access

View full text Add to dashboard Cite

“…Sequential recommender systems (SRS) play a crucial role in various domains, such as e-commerce [1,3,23,52,56], video [7,22], music [9,31] and social media [13,17]. The goal of these SRS is to predict the next item that a user is likely to interact with based on his/her historical behavior [44,46].…”

Section: Introductionmentioning

confidence: 99%

Is Contrastive Learning Necessary? A Study of Data Augmentation vs Contrastive Learning in Sequential Recommendation

Zhou,

Huang,

Xie

et al. 2024

Proceedings of the ACM Web Conference 2024

View full text Add to dashboard Cite

Sequential recommender systems (SRS) are designed to predict users' future behaviors based on their historical interaction data. Recent research has increasingly utilized contrastive learning (CL) to leverage unsupervised signals to alleviate the data sparsity issue in SRS. In general, CL-based SRS first augments the raw sequential interaction data by using data augmentation strategies and employs a contrastive training scheme to enforce the representations of those sequences from the same raw interaction data to be similar. Despite the growing popularity of CL, data augmentation, as a basic component of CL, has not received sufficient attention. This raises the question: Is it possible to achieve superior recommendation results solely through data augmentation? To answer this question, we benchmark eight widely used data augmentation strategies, as well as state-of-the-art CL-based SRS methods, on four real-world datasets under both warm-and cold-start settings. Intriguingly, the conclusion drawn from our study is that, certain data augmentation strategies can achieve similar or even superior performance compared with some CL-based methods, demonstrating the potential to significantly alleviate the data sparsity issue with fewer computational overhead. We hope that our study can further inspire more fundamental studies on the key functional components of complex CL techniques. Our processed datasets and codes are available at https://github.com/AIM-SE/DA4Rec.

show abstract

“…Although KD has steadily become a prevalent method for countering catastrophic forgetting, its integration into continual learning practices, its efficacy in overcoming forgetting, and its comprehensive influence on the field require thorough exploration. Most contemporary surveys on continual learning primarily investigate the field from various methodological categorizations [21], [8], [9], [15] and application domains [22], [23], [24], [25], yet there is a notable scarcity of reviews analyzing the field through the lens of specific techniques aimed at mitigating the issue of forgetting in continual learning. In this survey, we undertake a scrutiny of continual learning methods that implement KD, primarily within the realm of image classification tasks.…”

Section: Introductionmentioning

confidence: 99%

Continual Learning with Knowledge Distillation: A Survey

Li,

Su,

Zhang

et al. 2024

Preprint

View full text Add to dashboard Cite

The foremost challenge in continual learning is devising strategies to alleviate catastrophic forgetting, thereby preserving a model’s memory of prior knowledge while learning new tasks. Knowledge distillation, a form of data regularization, is garnering increasing attention in the field of continual learning for its ability to constrain a model’s discriminative power for previous tasks by emulating the outputs of old task models while learning new tasks, thus mitigating forgetting. This paper offers a comprehensive survey of continual learning methods employing knowledge distillation within the realm of image classification. We inductively categorize these methods according to the source of knowledge utilized and provide a detailed analysis of the distillation solutions they employ. Furthermore, given the characteristic inability of continual learning to access historical data, we introduce a novel taxonomy for continual learning approaches from the perspective of auxiliary data usage. In addition, we have conducted extensive experiments on CIFAR-100, TinyImageNet, and ImageNet-100 across nine knowledge distillation-integrated continual learning methods, deeply analyzing the role of knowledge distillation in different continual learning scenarios to alleviate model forgetting. Our substantial experimental evidence demonstrates that knowledge distillation can indeed reduce forgetting across most scenarios.

show abstract

A Survey on Incremental Update for Neural Recommender Systems

Cited by 3 publications

References 62 publications

News Recommendation With Word-Related Joint Topic Prediction

News Recommendation With Word-Related Joint Topic Prediction

Is Contrastive Learning Necessary? A Study of Data Augmentation vs Contrastive Learning in Sequential Recommendation

Continual Learning with Knowledge Distillation: A Survey

Contact Info

Product

Resources

About