Toward Pareto Efficient Fairness-Utility Trade-off in Recommendation through Reinforcement Learning

Abstract

The issue of fairness in recommendation is becoming increasingly essential as Recommender Systems (RS) touch and influence more and more people in their daily lives. In fairness-aware recommendation, most of the existing algorithmic approaches mainly aim at solving a constrained optimization problem by imposing a constraint on the level of fairness while optimizing the main recommendation objective, e.g., click through rate (CTR). While this alleviates the impact of unfair recommendations, the expected return of an approach may significantly compromise the recommendation accuracy due to the inherent trade-off between fairness and utility. This motivates us to deal with these conflicting objectives and explore the optimal trade-off between them in recommendation. One conspicuous approach is to seek a Pareto efficient/optimal solution to guarantee optimal compromises between utility and fairness. Moreover, considering the needs of real-world e-commerce platforms, it would be more desirable if we can generalize the whole Pareto Frontier, so that the decision-makers can specify any preference of one objective over another based on their current business needs. Therefore, in this work, we propose a fairness-aware recommendation framework using multi-objective reinforcement learning (MORL), called MoFIR (pronounced "more fair"), which is able to learn a single parametric representation for optimal recommendation policies over the space of all possible preferences. Specially, we modify traditional Deep Deterministic Policy Gradient (DDPG) by introducing conditioned network (CN) into it, which conditions the networks directly on these preferences and outputs Q-value-vectors. Experiments on several real-world recommendation datasets verify the superiority of our framework on both fairness metrics and recommendation measures when compared with all other baselines. We also extract the approximate Pareto Frontier on real-world datasets generated by MoFIR and compare to state-of-the-art fairness methods.

Fairness in Recommendation: A Survey

“…Some researches believe that recommendation is not only a prediction problem but a sequential decision problem, and suggest to model the recommendation problem as a Markov Decision Process (MDP) and solve the problem through Reinforcement Learning (RL) [8]. Some research on fairness in recommendation also follow the trend and consider RL methods to promote fairness through a long-term and dynamic perspective [73,75,89,93,120,[180][181][182]210]. In [73], the authors consider the dynamic item exposure fairness in recommender systems where the item popularity will change over time with recommendation actions and user feedback.…”

Fairness in Graph Mining: A Survey

“…To achieve a satisfying trade-off, a common strategy is to ensure the algorithm bearing Pareto optimality [106], [132], i.e., a state where either utility or fairness cannot be promoted without harming the other one. Graph mining algorithms also have the issue of utility-fairness trade-off [38], [39], [55]. For example, when the fairness-related regularization is added to the objective function of a specific graph analytical task, the solution of the regularized optimization problem often deviates from the solution that brings the best utility in the unregularized optimization problem.…”

Explainable Fairness in Recommendation

“…The issue of fairness in recommendation has received growing concerns as recommender systems touch and influence people's daily lives more deeply and profoundly [28,41,62]. Several recent works focusing on fairness quantification have found various types of bias and unfairness in recommendations, such as gender and race [12,41,69], item popularity [1,2,24,28], and user activeness [18,40].…”

“…The issue of fairness in recommendation has received growing concerns as recommender systems touch and influence people's daily lives more deeply and profoundly [28,41,62]. Several recent works focusing on fairness quantification have found various types of bias and unfairness in recommendations, such as gender and race [12,41,69], item popularity [1,2,24,28], and user activeness [18,40]. Meanwhile, the relevant methods for fair recommendation focusing on providing fair recommendation results based on pre-defined fairness, can be roughly divided into three categories: pre-processing, in-processing and post-processing algorithms [42].…”

“…Though their methods successfully provide explanations to the model disparities in simple tasks, they are not suitable for recommender systems where the model inputs could be extremely large and sparse, which may bring huge computational cost when calculating Shapley value for each feature. Furthermore, existing methods only partially answers the above question, since they only explain either utility or fairness alone, ignoring the fact that there is an inherent trade-off between fairness and utility, which has been demonstrated by several recent work both empirically and theoretically [28,33,34,44,45,71]. And this incomplete view may potentially downgrade the stringency of the method because explanations that have the same effect on model fairness may not have the same effect on model utility.…”