Bias and Debias in Recommender System: A Survey and Future Directions

Abstract

While recent years have witnessed a rapid growth of research papers on recommender system (RS), most of the papers focus on inventing machine learning models to better fit user behavior data. However, user behavior data is observational rather than experimental. This makes various biases widely exist in the data, including but not limited to selection bias, position bias, exposure bias, and popularity bias. Blindly fitting the data without considering the inherent biases will result in many serious issues, e.g., the discrepancy between offline evaluation and online metrics, hurting user satisfaction and trust on the recommendation service, etc. To transform the large volume of research models into practical improvements, it is highly urgent to explore the impacts of the biases and perform debiasing when necessary. When reviewing the papers that consider biases in RS, we find that, to our surprise, the studies are rather fragmented and lack a systematic organization. The terminology “bias” is widely used in the literature, but its definition is usually vague and even inconsistent across papers. This motivates us to provide a systematic survey of existing work on RS biases. In this paper, we first summarize seven types of biases in recommendation, along with their definitions and characteristics. We then provide a taxonomy to position and organize the existing work on recommendation debiasing. Finally, we identify some open challenges and envision some future directions, with the hope of inspiring more research work on this important yet less investigated topic. The summary of debiasing methods reviewed in this survey can be found at https://github.com/jiawei-chen/RecDebiasing.

Recommender systems, ground truth, and preference pollution

“…Unfortunately, in addition to a number of biases that online recommendations are known to manifest (Baeza‐Yates 2020; Chen et al. 2020), another inherent feature of recommender systems and the iterative user‐recommender interactions is that they pose significant challenges to both preference representativeness and preference independence characteristics of ground truth data. The former is not surprising, as recommender systems are explicitly designed to affect users’ consumption choices.…”

“…In particular, average ratings have a substantial human and social component, and they do not represent personalized information (as recognized, e.g., in Chen et al. 2020 under the characterization of a conformity bias).…”

“…There has already been a steady stream of studies about various biases in recommender systems (Baeza‐Yates 2020; Chen et al. 2020), resulting in a robust discussion of how some inherent aspects of recommender systems algorithms and interfaces—such as popularity bias (Abdollahpouri 2019; Abdollahpouri et al. 2017; Prawesh and Padmanabhan 2014), position bias (Collins et al.…”

On the User Behavior Leakage from Recommender System Exposure

“…The exposed items are selected by the recommender and thus can be regarded as a kind of system behavior data. Existing research regarding the exposure data mainly focuses on negative sampling from exposed items or exposure debiasing [5].…”

“…We adopt Recall to evaluate the attack performance. Let B𝑢 𝑖 @𝑘 denote the top-𝑘 inference output of the attack model 5 . Recall@𝑘 measures how many ground-truth user behaviors are included in B𝑢 𝑖 @𝑘, which is formulated as…”