KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems

Gao, Chongming; Li, Shijun; Lei, Wenqiang; Chen, Jiawei; Li, Biao; Jiang, Peng; He, Xiangnan; Mao, Jiaxin; Chua, Tat-Seng

doi:10.48550/arxiv.2202.10842

Cited by 3 publications

(6 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…KuaiEnv is created by us on the KuaiRec dataset 3 [12]. KuaiRec is a real-world dataset that contains a fully observed user-item interaction matrix, which means each user has viewed each video and then left feedback.…”

Section: Recommendation Environmentsmentioning

confidence: 99%

“…For pre-learning the user model 𝜙 𝑀 , we use the additional sparse user-video interactions in the big matrix. For the details of the data, please refer to the KuaiRec dataset [12].…”

Section: Recommendation Environmentsmentioning

confidence: 99%

“…In addition, we propose an interactive recommendation environment, KuaiEnv, for evaluating the problem. It is created based on the KuaiRec dataset that contains a fully filled user-item matrix (density: 99.6%) collected from the Kuaishou App 1 [12]. In this matrix, IRSs can be evaluated more faithfully since there are no missing values.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System

Gao¹,

Lei²,

Chen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

While personalization increases the utility of recommender systems, it also brings the issue of filter bubbles. E.g., if the system keeps exposing and recommending the items that the user is interested in, it may also make the user feel bored and less satisfied. Existing work studies filter bubbles in static recommendation, where the effect of overexposure is hard to capture. In contrast, we believe it is more meaningful to study the issue in interactive recommendation and optimize long-term user satisfaction. Nevertheless, it is unrealistic to train the model online due to the high cost. As such, we have to leverage offline training data and disentangle the causal effect on user satisfaction.To achieve this goal, we propose a counterfactual interactive recommender system (CIRS) that augments offline reinforcement learning (offline RL) with causal inference. The basic idea is to first learn a causal user model on historical data to capture the overexposure effect of items on user satisfaction. It then uses the learned causal user model to help the planning of the RL policy. To conduct evaluation offline, we innovatively create an authentic RL environment (KuaiEnv) based on a real-world fully observed user rating dataset. The experiments show the effectiveness of CIRS in bursting filter bubbles and achieving long-term success in interactive recommendation. The implementation of CIRS is available via https://github.com/chongminggao/CIRS-codes.

show abstract

Section: Recommendation Environmentsmentioning

confidence: 99%

“…For pre-learning the user model 𝜙 𝑀 , we use the additional sparse user-video interactions in the big matrix. For the details of the data, please refer to the KuaiRec dataset [12].…”

Section: Recommendation Environmentsmentioning

confidence: 99%

See 1 more Smart Citation

CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System

Gao¹,

Lei²,

Chen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We have conducted our experiments on three publicly-available real-world datasets for recommendation: MovieLens-1m 5 , Amazon-14core 6 , and KuaiRec-binary 7 [8]. Among them, MovieLens is a well-known classic which has been extensively studied, while KuaiRec is a fairly new one that just emerged earlier this year.…”

Section: Data and Codementioning

confidence: 99%

Forgetting Fast in Recommender Systems

Liu¹,

Wan²,

Wang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Users of a recommender system may want part of their data being deleted, not only from the data repository but also from the underlying machine learning model, for privacy or utility reasons. Such right-to-be-forgotten requests could be fulfilled by simply retraining the recommendation model from scratch, but that would be too slow and too expensive in practice. In this paper, we investigate fast machine unlearning techniques for recommender systems that can remove the effect of a small amount of training data from the recommendation model without incurring the full cost of retraining. A natural idea to speed this process up is to fine-tune the current recommendation model on the remaining training data instead of starting from a random initialization. This warm-start strategy indeed works for neural recommendation models using standard 1st-order neural network optimizers (like AdamW). However, we have found that even greater acceleration could be achieved by employing 2nd-order (Newton or quasi-Newton) optimization methods instead. To overcome the prohibitively high computational cost of 2nd-order optimizers, we propose a new recommendation unlearning approach AltEraser which divides the optimization problem of unlearning into many small tractable sub-problems.Extensive experiments on three real-world recommendation datasets show promising results of AltEraser in terms of consistency (forgetting thoroughness), accuracy (recommendation effectiveness), and efficiency (unlearning speed). To our knowledge, this work represents the first attempt at fast approximate machine unlearning for state-of-the-art neural recommendation models.

show abstract

“…The fundamental problem lies in the fact that we have no knowledge about the massive missing interactions in the offline data [6].…”

Section: Open Problem: Collecting Random Datamentioning

confidence: 99%

KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos

Gao,

Li,

Zhang

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Recommender systems deployed in real-world applications can have inherent exposure bias, which leads to the biased logged data plaguing the researchers. A fundamental way to address this thorny problem is to collect users' interactions on randomly expose items, i.e., the missing-at-random data. A few works have asked certain users to rate or select randomly recommended items, e.g., Yahoo! [16], Coat [22], and OpenBandit [19]. However, these datasets are either too small in size or lack key information, such as unique user ID or the features of users/items. In this work, we present KuaiRand, an unbiased sequential recommendation dataset containing millions of intervened interactions on randomly exposed videos, collected from the video-sharing mobile App, Kuaishou. Different from existing datasets, KuaiRand records 12 kinds of user feedback signals (e.g., click, like, and view time) on randomly exposed videos inserted in the recommendation feeds in two weeks. To facilitate model learning, we further collect rich features of users and items as well as users' behavior history. By releasing this dataset, we enable the research of advanced debiasing large-scale recommendation scenarios for the first time. Also, with its distinctive features, KuaiRand can support various other research directions such as interactive recommendation, long sequential behavior modeling, and multi-task learning. The dataset and its news will be available at https://kuairand.com. CCS CONCEPTS• Information systems → Recommender systems.

show abstract

KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems

Cited by 3 publications

References 46 publications

CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System

CIRS: Bursting Filter Bubbles by Counterfactual Interactive Recommender System

Forgetting Fast in Recommender Systems

KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos

Contact Info

Product

Resources

About