Position Bias in Recommender Systems for Digital Libraries

Collins, Andrew; Tkaczyk, Dominika; Aizawa, Akiko; Beel, Joeran

doi:10.1007/978-3-319-78105-1_37

Cited by 35 publications

(24 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Para contornar esse problema, os dados passam por um processo de eliminação de bias, ou ainda eliminação de enviesamento tendencioso, antes de serem usados. A eliminação de biasé por sí só um processo que pode trazer diversos desafios de pesquisa [Krishnan et al 2014, Collins et al 2018, Abdollahpouri et al 2019.…”

Section: Arquiteturaunclassified

Jusbrasil e os Desafios Tecnológicos para Facilitar e Aprimorar o Acesso à Justiça

Moura¹,

Costa²,

Jordão³

et al. 2021

Anais Do XLVIII Seminário Integrado De Software E Hardware (SEMISH 2021)

View full text Add to dashboard Cite

Este artigo apresenta uma breve introdução a alguns dos desafios tecnológicos enfrentados na empresa Jusbrasil em sua missão de facilitar o acesso à justiça no Brasil. A Jusbrasil busca unir direito e tecnologia para que a justiça ultrapasse as fronteiras dos tribunais e chegue às casas de qualquer cidadão ou cidadã. Milhões de pessoas acessam a plataforma da empresa atualmente, havendo acesso de mais de 900 mil advogados. Por outro lado, nossa base de informação possui bilhões de documentos contendo artefatos relacionados ao direito no Brasil. Essa base de informação traz oportunidades para o desenvolvimento de produtos inteligentes, tais como sistemas de busca eficazes e eficientes, ferramentas de estruturação e tratamento da informação, mineração de dados e sistemas de recomendação, dentre outros. Neste trabalho, discutimos e apresentamos para a comunidade acadêmica alguns dos desafios tecnológicos enfrentados pela empresa no dia-a-dia.

show abstract

Section: Arquiteturaunclassified

Jusbrasil e os Desafios Tecnológicos para Facilitar e Aprimorar o Acesso à Justiça

Moura¹,

Costa²,

Jordão³

et al. 2021

Anais Do XLVIII Seminário Integrado De Software E Hardware (SEMISH 2021)

View full text Add to dashboard Cite

show abstract

“…More recently, Kate Crawford's presentation related to research on the fairness of machine learning algorithms attracted more attention from the machine learning community on the problem of algorithmic bias [82]. Some work studied biases emerging due to item popularity [117][118][119][120]. A recent work studied bias that is due to the assimilation bias in recommender systems [121].…”

Section: Plos Onementioning

confidence: 99%

Evolution and impact of bias in human and machine learning algorithm interaction

2020

View full text Add to dashboard Cite

Traditionally, machine learning algorithms relied on reliable labels from experts to build predictions. More recently however, algorithms have been receiving data from the general population in the form of labeling, annotations, etc. The result is that algorithms are subject to bias that is born from ingesting unchecked information, such as biased samples and biased labels. Furthermore, people and algorithms are increasingly engaged in interactive processes wherein neither the human nor the algorithms receive unbiased data. Algorithms can also make biased predictions, leading to what is now known as algorithmic bias. On the other hand, human's reaction to the output of machine learning methods with algorithmic bias worsen the situations by making decision based on biased information, which will probably be consumed by algorithms later. Some recent research has focused on the ethical and moral implication of machine learning algorithmic bias on society. However, most research has so far treated algorithmic bias as a static factor, which fails to capture the dynamic and iterative properties of bias. We argue that algorithmic bias interacts with humans in an iterative manner, which has a long-term effect on algorithms' performance. For this purpose, we present an iterated-learning framework that is inspired from human language evolution to study the interaction between machine learning algorithms and humans. Our goal is to study two sources of bias that interact: the process by which people select information to label (human action); and the process by which an algorithm selects the subset of information to present to people (iterated algorithmic bias mode). We investigate three forms of iterated algorithmic bias (personalization filter, active learning, and random) and how they affect the performance of machine learning algorithms by formulating research questions about the impact of each type of bias. Based on statistical analyses of the results of several controlled experiments, we found that the three different iterated bias modes, as well as initial training data class imbalance and human action, do affect the models learned by machine learning algorithms. We also found that iterated filter bias, which is prominent in personalized user interfaces, can lead to more inequality in estimated relevance and to a limited human ability to discover relevant data. Our findings indicate that the relevance blind spot (items from the testing set whose predicted relevance probability is less than 0.5 and who thus risk being hidden from humans) amounted to 4% of all relevant items when using a content-based filter

show abstract

“…Implicit feedback bias, as it pertains to recommender systems has not been deeply explored, although implicit feedback is heavily used in practice, often in an online fashion [31,44]. There are some empirical case studies on position bias in recommender systems [15]. Several works focus on unbiased offline evaluation [19,39] without studying how to learn unbiased models.…”

Section: Related Workmentioning

confidence: 99%

“…However, implicit feedback is usually biased. Position bias is a well-known and often-studied type of bias [15,19]. This bias focuses on the fact that users tend to react more favorably to the items at more visible positions [43].…”

Section: Introductionmentioning

confidence: 99%

Attribute-based Propensity for Unbiased Learning in Recommender Systems

Qin

Chen

Metzler

et al. 2020

Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery &Amp; Data Mining

View full text Add to dashboard Cite

Many modern recommender systems train their models based on a large amount of implicit user feedback data. Due to the inherent bias in this data (e.g., position bias), learning from it directly can lead to suboptimal models. Recently, unbiased learning was proposed to address such problems by leveraging counterfactual techniques like inverse propensity weighting (IPW). In these methods, propensity scores estimation is usually limited to item's display position in a single user interface (UI). In this paper, we generalize the traditional position bias model to an attribute-based propensity framework. Our methods estimate propensity scores based on offline data and allow propensity estimation across a broad range of implicit feedback scenarios, e.g., feedback beyond recommender system UI. We demonstrate this by applying this framework to three real-world large-scale recommender systems in Google Drive that serve millions of users. For each system, we conduct both offline and online evaluation. Our results show that the proposed framework is able to significantly improve upon strong production baselines across a diverse range of recommendation item types (documents, people-document pairs, and queries), UI layouts (horizontal, vertical, and grid layouts), and underlying learning algorithms (gradient boosted decision trees and neural networks), all without the need to intervene and degrade the user experience. The proposed models have been deployed in the production systems with ease since no serving infrastructure change is needed.

show abstract

Position Bias in Recommender Systems for Digital Libraries

Cited by 35 publications

References 17 publications

Jusbrasil e os Desafios Tecnológicos para Facilitar e Aprimorar o Acesso à Justiça

Jusbrasil e os Desafios Tecnológicos para Facilitar e Aprimorar o Acesso à Justiça

Evolution and impact of bias in human and machine learning algorithm interaction

Attribute-based Propensity for Unbiased Learning in Recommender Systems

Contact Info

Product

Resources

About