Trust in Artificial Intelligence: Comparing trust processes between human and automated trustees in light of unfair bias

Abstract

Introducing automated systems based on artificial intelligence and machine learning for ethically sensitive decision tasks requires investigating of trust processes in relation to such tasks. In an example of such a task (personnel selection), this study investigates trustworthiness, trust, and reliance in light of a trust violation relating to ethical standards and a trust repair intervention. Specifically, participants evaluated applicant preselection outcomes by either a human or an automated system across twelve personnel selection tasks. We additionally varied information regarding imperfection of the human and automated system. In task rounds five through eight, the preselected applicants were predominantly male, thus constituting a trust violation due to a violation of ethical standards. Before task round nine, participants received an excuse for the biased preselection (i.e., a trust repair intervention). Results showed that participants initially perceived automated systems to be less trustworthy, and had less intention to trust automated systems. Specifically, participants perceived systems to be less able, and flexible, but also less biased – a result that was sustained even in light of unfair bias. Furthermore, in regard to the automated system the trust violation and the trust repair intervention had weaker effects. Those effects were partly stronger when highlighting imperfection for the automated system. We conclude that it is crucial to investigate trust processes in relation to automated systems in ethically sensitive domains such as personnel selection as insights from classical areas of automation might not translate to application contexts where ethical standards are central to trust processes.

Dear Computer on My Desk, Which Candidate Fits Best? An Assessment of Candidates’ Perception of Assessment Quality When Using AI in Personnel Selection

“…Of note, system reliability is only relevant in association with the other dimensions of AI, indicating that the mere mentioning, that AI is reliable does not improve assessment quality perceptions. Potentially, this may be due to a lack of information on the functioning of the different AI methods in this study (see Langer et al, 2021b ).…”

Should I Trust the Artificial Intelligence to Recruit? Recruiters’ Perceptions and Behavior When Faced With Algorithm-Based Recommendation Systems During Resume Screening

“…In this research, we analyzed participants’ trustworthiness and subsequent behaviors (i.e., scoring and ranking application forms) after being exposed to recommendations. To enhance ecological validity of our results, we selected a sample of experienced professionals involved in the recruitment process, in contrast with previous research on trust in selection context, in which participants involved were mainly students or working adults ( Diab et al, 2011 ; Dietvorst et al, 2015 ; Dietvorst, 2016 ; Lee, 2018 ; Langer et al, 2021a ) and only a few studies used participants actually involved in recruitment tasks in their organization ( Oberst et al, 2020 ). We selected participants from an online panel owned by a specialized company, Panelabs, which gives access to a panel that contains 500,000 French participants and provides a high level of quality control.…”

“…Most studies revealed a tendency to disuse automation and highlight an aversion behavior. For example, Langer et al (2021a) concluded that participants in a personnel selection context perceived the human trustee as more trustworthy than an automated trustee. The authors explained these results by noting that human ability is necessary to complete tasks that involved ethical issues.…”

“…However, the authors also note the scarcity of field studies that assess trust in ADSSs (which they refer to as “embedded AI”) in organizational settings. As mentioned previously, this scarcity is of particular concern for personnel selection, in which the market for predictive hiring tools is growing fast but academic research remains at an early stage ( Campion et al, 2016 ; Langer et al, 2019b , 2021a ; Tambe et al, 2019 ; Oberst et al, 2020 ). In personnel selection, the stakes are high because it involves ethic, legal, psychological, and strategic issues in organizations.…”

“…Moreover, human intervention remains important: the recruitment process is not fully automated, which leaves room for the user to choose whether to use ADSSs. As Langer et al (2021a , p. 3) note, this latitude for choice means that managers must assess the relative trustworthiness of humans and automated systems to decide whether to rely on ADSSs for a given task. A better understanding of the mechanisms governing this trust and its consequences on decision process for recruiters remains an important issue.…”