Achieving Fairness in the Stochastic Multi-Armed Bandit Problem

Patil, Vishakha; Ghalme, Ganesh; Nair, Vineet; Narahari, Y.

doi:10.1609/aaai.v34i04.5986

Cited by 47 publications

(35 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [93] investigated specifically the dynamics of group qualification rates [61,79] under the more general partially-observed MDP setting. Moreover, there also exist some studies of fairness-aware (contextual) multi-armed bandits [5,28,29,44,73] -a simplified form of reinforcement learning -where the fairness constraint is defined as a minimum rate at which a task/resource is assigned to a user [12,13,48,62,85].…”

Section: Dynamic Recommendation Fairnessmentioning

confidence: 99%

Recommendation Fairness: From Static to Dynamic

Zhang¹,

Wang²

2021

Preprint

View full text Add to dashboard Cite

Driven by the need to capture users' evolving interests and optimize their long-term experiences, more and more recommender systems have started to model recommendation as a Markov decision process and employ reinforcement learning to address the problem. Shouldn't research on the fairness of recommender systems follow the same trend from static evaluation and one-shot intervention to dynamic monitoring and non-stop control? In this paper, we portray the recent developments in recommender systems first and then discuss how fairness could be baked into the reinforcement learning techniques for recommendation. Moreover, we argue that in order to make further progress in recommendation fairness, we may want to consider multi-agent (game-theoretic) optimization, multi-objective (Pareto) optimization, and simulationbased optimization, in the general framework of stochastic games. CCS CONCEPTS• Information systems → Recommender systems; • Computing methodologies → Reinforcement learning.

show abstract

Section: Dynamic Recommendation Fairnessmentioning

confidence: 99%

Recommendation Fairness: From Static to Dynamic

Zhang¹,

Wang²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Fairness under Other Temporal Models A series of works focus on study fairness in the setting of online learning (Blum et al, 2018;Bechavod et al, 2019;Gupta and Kamble, 2019), multi-armed bandit (Joseph et al, 2016;Patil et al, 2020;, and one-step feedback model (Liu et al, 2018;Hu and Chen, 2018;Kannan et al, 2019). In particular, Hashimoto et al (2018) show that empirical risk minimization amplifies representation disparity over time with a low group retention rate for the underrepresented group.…”

Section: Related Workmentioning

confidence: 99%

Towards Return Parity in Markov Decision Processes

Chi¹,

Shen²,

Dai³

et al. 2021

Preprint

View full text Add to dashboard Cite

Algorithmic decisions made by machine learning models in high-stakes domains may have lasting impacts over time. Unfortunately, naive applications of standard fairness criterion in static settings over temporal domains may lead to delayed and adverse effects. To understand the dynamics of performance disparity, we study a fairness problem in Markov decision processes (MDPs). Specifically, we propose return parity, a fairness notion that requires MDPs from different demographic groups that share the same state and action spaces to achieve approximately the same expected time-discounted rewards. We first provide a decomposition theorem for return disparity, which decomposes the return disparity of any two MDPs into the distance between group-wise reward functions, the discrepancy of group policies, and the discrepancy between state visitation distributions induced by the group policies. Motivated by our decomposition theorem, we propose algorithms to mitigate return disparity via learning a shared group policy with state visitation distributional alignment using integral probability metrics. We conduct experiments to corroborate our results, showing that the proposed algorithm can successfully close the disparity gap while maintaining the performance of policies on two real-world recommender system benchmark datasets.

show abstract

“…Manshadi et al [39] studied fair online rationing such that each arriving agent can receive a fair share of resources proportional to its demand. The fairness issue has been studied in other domains/applications as well, see, e.g., online selection of candidates [40], influence maximization [41], banditbased online learning [42][43][44], online resource allocation [45,46], and classification [47].…”

Section: Other Related Workmentioning

confidence: 99%

Fairness Maximization among Offline Agents in Online-Matching Markets

Ma¹,

Xu²,

Xu³

2021

Preprint

View full text Add to dashboard Cite

Matching markets involve heterogeneous agents (typically from two parties) who are paired for mutual benefit. During the last decade, matching markets have emerged and grown rapidly through the medium of the Internet. They have evolved into a new format, called Online Matching Markets (OMMs), with examples ranging from crowdsourcing to online recommendations to ridesharing. There are two features distinguishing OMMs from traditional matching markets. One is the dynamic arrival of one side of the market: we refer to these as online agents while the rest are offline agents. Examples of online and offline agents include keywords (online) and sponsors (offline) in Google Advertising; workers (online) and tasks (offline) in Amazon Mechanical Turk (AMT); riders (online) and drivers (offline when restricted to a short time window) in ridesharing. The second distinguishing feature of OMMs is the real-time decision-making element. However, studies have shown that the algorithms making decisions in these OMMs leave disparities in the match rates of offline agents. For example, tasks in neighborhoods of low socioeconomic status rarely get matched to gig workers, and drivers of certain races/genders get discriminated against in matchmaking. In this paper, we propose online matching algorithms which optimize for either individual or group-level fairness among offline agents in OMMs. We present two linear-programming (LP) based sampling algorithms, which achieve online competitive ratios at least 0.725 for individual fairness maximization (IFM) and 0.719 for group fairness maximization (GFM), respectively. There are two key ideas helping us break the barrier of 1 − 1/e. One is boosting, which is to adaptively re-distribute all sampling probabilities only among those offline available neighbors for every arriving online agent. The other is attenuation, which aims to balance the matching probabilities among offline agents with different mass allocated by the benchmark LP. We conduct extensive numerical experiments and results show that our boosted version of sampling algorithms are not only conceptually easy to implement but also highly effective in practical instances of fairness-maximization-related models.

show abstract

Achieving Fairness in the Stochastic Multi-Armed Bandit Problem

Cited by 47 publications

References 12 publications

Recommendation Fairness: From Static to Dynamic

Recommendation Fairness: From Static to Dynamic

Towards Return Parity in Markov Decision Processes

Fairness Maximization among Offline Agents in Online-Matching Markets

Contact Info

Product

Resources

About