“There Is Not Enough Information”: On the Effects of Explanations on Perceptions of Informational Fairness and Trustworthiness in Automated Decision-Making

Abstract

Automated decision systems (ADS) are increasingly used for consequential decision-making. These systems often rely on sophisticated yet opaque machine learning models, which do not allow for understanding how a given decision was arrived at. In this work, we conduct a human subject study to assess people's perceptions of informational fairness (i.e., whether people think they are given adequate information on and explanation of the process and its outcomes) and trustworthiness of an underlying ADS when provided with varying types of information about the system. More specifically, we instantiate an ADS in the area of automated loan approval and generate different explanations that are commonly used in the literature. We randomize the amount of information that study participants get to see by providing certain groups of people with the same explanations as others plus additional explanations. From our quantitative analyses, we observe that different amounts of information as well as people's (self-assessed) AI literacy significantly influence the perceived informational fairness, which, in turn, positively relates to perceived trustworthiness of the ADS. A comprehensive analysis of qualitative feedback sheds light on people's desiderata for explanations, among which are (i) consistency (both with people's expectations and across different explanations), (ii) disclosure of monotonic relationships between features and outcome, and (iii) actionability of recommendations. CCS CONCEPTS• Human-centered computing → Human computer interaction (HCI); • Computing methodologies → Machine learning; • Information systems → Decision support systems.

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

“…Thus, procedural fairness can be increased even if the decision outcome is biased by using appropriate input features and ensuring people can appeal an ADM decision. We further find that five studies also included interactional fairness, and six studies investigated informational fairness (Acikgoz et al, 2020;Binns et al, 2018;Schoeffer et al, 2021). These studies zoom into the decision process and emphasize its social aspects, such as treating individuals with respect and receiving an explanation for an ADM decision (Schlicker et al, 2021).…”

“…After being asked, "Who would, according to you, make a fairer decision: a human or artificial intelligence/computer?," 54% of respondents answered that they believed AI makes fairer decisions (compared to 33% for humans) (Helberger et al, 2020). Schoeffer et al (2021) investigated open-ended questions and also found that respondents perceived ADM as less biased than HDM. Other studies support this general finding in university admission decisions (Marcinkowski et al, 2020) and algorithmic work assignments (Bai et al, 2020).…”

“…Moreover, several studies showed that different explanation styles (e.g. case-based, sensitivity-based, demographic-based, input influencebased) affected respondents' perceived fairness differently (Binns et al, 2018;Dodge et al, 2019;Schoeffer et al, 2021;Shulner-Tal et al, 2022).…”

“…While participating in a workshop about ethical AI raised awareness about algorithmic fairness (Kasinidou et al, 2021b), understanding the mathematical definition of the fairness concept led respondents to reject the fairness concept (Saha et al, 2020). While some other studies indicated that higher levels of AI literacy were associated with lower levels of perceived algorithmic fairness (Lee and Baykal, 2017;Schoeffer et al, 2021), Araujo et al (2020) found that the effects ran in the opposite direction.…”

Flexible and Robust Counterfactual Explanations with Minimal Satisfiable Perturbations

“…Counterfactual explanation (CFE) [44] aims to identify minimal changes required to modify the input to achieve a desired prediction and provides insights into why a model produces a certain prediction instead of the desired one. CFEs can help understand the underlying logic of certain predictions [31], detect the inherent model bias for fairness [19], and provide suggestions to users who receive adverse predictions [17,39]. Therefore, CFEs can be adopted in broad applications of healthcare, finance, education, justice, and other domains.…”