Empowering News Recommendation with Pre-trained Language Models

Wu, Chuhan; Wu, Fangzhao; Qi, Tao; Huang, Yudong

doi:10.1145/3404835.3463069

Cited by 98 publications

(43 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Personalized news recommendation is a critical way to personalize news distribution and alleviate the information overload problem. Multiple news recommendation methods have been proposed recently [27,29,30,40,42,43,46]. Generally, there are three core components in news recommendation methods: news model, user model, and click prediction module.…”

Section: Related Work 21 Personalized News Recommendationmentioning

confidence: 99%

“…Wu et al [45] use the combination of multi-head selfattention and additive attention to learn news representations. Wu et al [46] apply pre-trained language model in the news model to empower its semantic understanding ability. The user model is used to learn user representations from users' historical clicked news representations.…”

Section: Related Work 21 Personalized News Recommendationmentioning

confidence: 99%

“…representation, which can be implemented by dot product [46], cosine similarity [14], or MLP network [41].…”

Section: Related Work 21 Personalized News Recommendationmentioning

confidence: 99%

See 2 more Smart Citations

UA-FedRec: Untargeted Attack on Federated News Recommendation

Yi¹,

Wu²,

Zhu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

News recommendation is critical for personalized news distribution. Federated news recommendation enables collaborative model learning from many clients without sharing their raw data. It is promising for privacy-preserving news recommendation. However, the security of federated news recommendation is still unclear. In this paper, we study this problem by proposing an untargeted attack called UA-FedRec. By exploiting the prior knowledge of news recommendation and federated learning, UA-FedRec can effectively degrade the model performance with a small percentage of malicious clients. First, the effectiveness of news recommendation highly depends on user modeling and news modeling. We design a news similarity perturbation method to make representations of similar news farther and those of dissimilar news closer to interrupt news modeling, and propose a user model perturbation method to make malicious user updates in opposite directions of benign updates to interrupt user modeling. Second, updates from different clients are typically aggregated by weighted-averaging based on their sample sizes. We propose a quantity perturbation method to enlarge sample sizes of malicious clients in a reasonable range to amplify the impact of malicious updates. Extensive experiments on two real-world datasets show that UA-FedRec can effectively degrade the accuracy of existing federated news recommendation methods, even when defense is applied. Our study reveals a critical security issue in existing federated news recommendation systems and calls for research efforts to address the issue. CCS CONCEPTS• Information systems → Collaborative filtering.

show abstract

Section: Related Work 21 Personalized News Recommendationmentioning

confidence: 99%

Section: Related Work 21 Personalized News Recommendationmentioning

confidence: 99%

See 1 more Smart Citation

UA-FedRec: Untargeted Attack on Federated News Recommendation

Yi¹,

Wu²,

Zhu³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Wu et al [156] proposed to learn news representations from news titles via a combination of multi-head self-attention and additive attention networks. Wu et al [161] studied to use pre-trained language models to encode news texts. These deep learning-based news modeling methods can automatically learn informative news representations without the need of manual feature engineering, and they can usually better understand news content than traditional feature-based methods.…”

Section: News Modelingmentioning

confidence: 99%

“…These methods usually learn news representations based on shallow text models and non-contextualized word embeddings such as GloVe [119], which may be insufficient to capture the deep semantic information in news. Pre-trained language models (PLMs) such as BERT [27] have been greatly successful in the NLP field, and a few recent works explore to empower news modeling with PLMs [161,167]. For example, PLM-NR [161] uses different PLMs to empower English and multilingual news recommendation, and the online flight results in Microsoft News showed notable performance improvement.…”

Section: Deep Learning-based News Modelingmentioning

confidence: 99%

Personalized News Recommendation: Methods and Challenges

Wu¹,

Wu²,

Huang³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Personalized news recommendation is an important technique to help users find their interested news information and alleviate their information overload. It has been extensively studied over decades and has achieved notable success in improving users' news reading experience. However, there are still many unsolved problems and challenges that need to be further studied. To help researchers master the advances in personalized news recommendation over the past years, in this paper we present a comprehensive overview of personalized news recommendation. Instead of following the conventional taxonomy of news recommendation methods, in this paper we propose a novel perspective to understand personalized news recommendation based on its core problems and the associated techniques and challenges. We first review the techniques for tackling each core problem in a personalized news recommender system and the challenges they face. Next, we introduce the public datasets and evaluation metrics used for personalized news recommendation. We then discuss the key points on improving the responsibility of personalized news recommender systems. Finally, we raise several research directions that are worth investigating in future. This paper can provide up-to-date and comprehensive views to help readers understand the personalized news recommendation field. We hope this paper can facilitate research on personalized news recommendation and as well as related fields in natural language processing and data mining.

show abstract

An Empirical Study on News Recommendation in Multiple Domain Settings

Haruta

Kurokawa

2022

Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

View full text Add to dashboard Cite

Empowering News Recommendation with Pre-trained Language Models

Cited by 98 publications

References 12 publications

UA-FedRec: Untargeted Attack on Federated News Recommendation

UA-FedRec: Untargeted Attack on Federated News Recommendation

Personalized News Recommendation: Methods and Challenges

An Empirical Study on News Recommendation in Multiple Domain Settings

Contact Info

Product

Resources

About