Using Pareto simulated annealing to address algorithmic bias in machine learning

Abstract

Algorithmic bias arises in machine learning when models that may have reasonable overall accuracy are biased in favor of ‘good’ outcomes for one side of a sensitive category, for example gender or race. The bias will manifest as an underestimation of good outcomes for the under-represented minority. In a sense, we should not be surprised that a model might be biased when it has not been ‘asked’ not to be; reasonable accuracy can be achieved by ignoring the under-represented minority. A common strategy to address this issue is to include fairness as a component in the learning objective. In this paper, we consider including fairness as an additional criterion in model training and propose a multi-objective optimization strategy using Pareto Simulated Annealing that optimizes for both accuracy and underestimation bias. Our experiments show that this strategy can identify families of models with members representing different accuracy/fairness tradeoffs. We demonstrate the effectiveness of this strategy on two synthetic and two real-world datasets.

Special issue on evolutionary machine learning

“…In the first paper Using Pareto simulated annealing to address algorithmic bias in machine learning by Blanzeisky and Cunningham (2022), the authors utilize fairness within the learning objective to mitigate algorithmic bias and propose a multi-objective optimization strategy using pareto simulated annealing that considers both accuracy and bias. Blanzeisky and Cunningham evaluate their proposed algorithm using 4 classification datasets.…”