Check the box! How to deal with automation bias in AI-based personnel selection

Abstract

Artificial Intelligence (AI) as decision support for personnel preselection, e.g., in the form of a dashboard, promises a more effective and fairer selection process. However, AI-based decision support systems might prompt decision makers to thoughtlessly accept the system’s recommendation. As this so-called automation bias contradicts ethical and legal requirements of human oversight for the use of AI-based recommendations in personnel preselection, the present study investigates strategies to reduce automation bias and increase decision quality. Based on the Elaboration Likelihood Model, we assume that instructing decision makers about the possibility of system errors and their responsibility for the decision, as well as providing an appropriate level of data aggregation should encourage decision makers to process information systematically instead of heuristically. We conducted a 3 (general information, information about system errors, information about responsibility) x 2 (low vs. high aggregated data) experiment to investigate which strategy can reduce automation bias and enhance decision quality. We found that less automation bias in terms of higher scores on verification intensity indicators correlated with higher objective decision quality, i.e., more suitable applicants selected. Decision makers who received information about system errors scored higher on verification intensity indicators and rated subjective decision quality higher, but decision makers who were informed about their responsibility, unexpectedly, did not. Regarding aggregation level of data, decision makers of the highly aggregated data group spent less time on the level of the dashboard where highly aggregated data were presented. Our results show that it is important to inform decision makers who interact with AI-based decision-support systems about potential system errors and provide them with less aggregated data to reduce automation bias and enhance decision quality.

In the AI of the Beholder—A Qualitative Study of HR Professionals’ Beliefs about AI-Based Chatbots and Decision Support in Candidate Pre-Selection

“…Experts are convinced that AI will offer benefits to businesses, and AI is superior to humans for certain tasks (e.g., information retrieval) and can be considered "state of the art" (Lücke 2019). However, it is assumed that individual variables of HR professionals influence the interaction with AI systems and thus also the decision quality (Kupfer et al 2023). Our results show that HR professionals' respective beliefs determine perceived use cases and barriers and thus have an impact on the actual use of AI.…”