Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models

Abstract

This study examines issues of algorithmic fairness in the context of systems that inform tax audit selection by the United States Internal Revenue Service (IRS). While the field of algorithmic fairness has developed primarily around notions of treating like individuals alike, we instead explore the concept of vertical equity-appropriately accounting for relevant differences across individuals-which is a central component of fairness in many public policy settings. Applied to the design of the U.S. individual income tax system, vertical equity relates to the fair allocation of tax and enforcement burdens across taxpayers of different income levels. Through a unique collaboration with the Treasury Department and IRS, we use access to detailed, anonymized individual taxpayer microdata, risk-selected audits, and random audits from 2010-14 to study vertical equity in tax administration. In particular, we assess how the adoption of modern machine learning methods for selecting taxpayer audits may affect vertical equity. Our paper makes four contributions.First, we show how the adoption of more flexible machine learning (classification) methods-as opposed to simpler models-shapes vertical equity by shifting audit burdens from high to middle-income taxpayers. Second, given concerns about high audit rates of low-income taxpayers, we investigate how existing algorithmic fairness techniques would change the audit distribution. We find that such methods can mitigate some disparities across income buckets, but that these come at a steep cost to performance. Third, we show that the choice of whether to treat risk of underreporting as a classification or regression problem is highly consequential.Moving from a classification approach to a regression approach to predict the expected magnitude of underreporting shifts the audit burden substantially toward high income individuals, while increasing revenue. Last, we investigate the role of differential audit cost in shaping the distribution of audits. Audits of lower income taxpayers, for instance, are typically conducted by mail and hence pose much lower cost to the IRS. We show that a narrow focus on return-on-investment can undermine vertical equity. Our results have implications for ongoing policy debates and the design of algorithmic tools across the public sector.

Multi-Target Multiplicity: Flexibility and Fairness in Target Specification under Resource Constraints

“…Recent work on algorithmic fairness has noted the importance of considering resource constraints. For instance, Black et al [3] discuss how the increased cost of auditing more complex tax filings can lead to prediction-based auditing strategies that disproportionately focus on lower income earners. Other work has emphasized the importance of considering resource constraints in the context of algorithmic fairness in healthcare [36] and business analytics [10].…”

“…Thus far our focus has been on multiplicity and identifying flippable points-those whose decision depends on the particular model chosen among the set G of good models. 3 In practice, however, we may be equally interested in unflippable points. As prior work has pointed out, the presence of multiplicity raises concerns about arbitrariness: What justification can you offer someone who receives an adverse decision from the chosen model when there may exist another good model that would have given them a favorable decision [5]?…”

“…X train X tune X holdout Y (1) train Y (2) train Y (3) train M (1) M (2) M (3) Step 1: Train one model for each target Y (k) train given X train…”

“…⍺ (1) , ⍺ (2) , ⍺ (3) Step 4: Use each model M (k) to predict Ŷ (k) holdout from X holdout , weight according to ⍺ Training data is used to fit separate models for each target variable. In the tune phase, the fitted models are used to forecast each target variable, and the index model MIP is run to find a fairness-maximizing weighted combination of the targets.…”

Estimating Racial Disparities When Race is Not Observed

“…The identification and estimation of racial disparities is of paramount importance to researchers, policymakers and organizations in a variety of areas including public health (Van Ryn and Fu, 2003;Williams and Jackson, 2005), employment (Conway and Roberts, 1983;Greene, 1984), voting (Gay, 2001;Hajnal and Trounstine, 2005;Barreto, 2007), criminal justice (Berk et al, 2021;Chouldechova, 2017;Dressel and Farid, 2018), economic policy (Brown, 2022), taxation (Black et al, 2022;Elzayn et al, 2023), housing (Kermani and Wong, 2021), lending (Chen, 2018), and technology and fairness (Alao et al, 2021). Within the U.S. government, efforts to identify and remedy racial disparities have taken on greater urgency with the recent issuance of Executive Order 13985, which in part directs agencies to conduct equity assessments by developing appropriate methodology.…”