Measuring Automation Bias and Complacency in an X-Ray Screening Task

Davis, Jacob; Atchley, Andrew; Smitherman, Hannah; Simon, Hailey; Tenhundfeld, Nathan L.

doi:10.1109/sieds49339.2020.9106670

Cited by 5 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automation bias has been found to be a serious problem in contexts of AI-based decision support systems ( Mosier and Skitka, 1999 ; Goddard et al, 2012 ; Lyell et al, 2018 ; Davis et al, 2020 ), and violates ethical recommendations ( Hunkenschroer and Luetge, 2022 ) as well as legal requirements like Article 22 of the GDPR ( The European Parliament and the Council of the European Union, 2016 ) or the EU AI act ( European Commission, 2021 ) that call for human oversight. Studies that previously examined AI-based personnel preselection tools from the perspective of decision makers have not yet addressed automation bias.…”

Section: Discussionmentioning

confidence: 99%

“…Automation bias describes the tendency of people to thoughtlessly accept an automated decision or recommendation. Thus far, automation bias and its negative outcomes have primarily been investigated in aviation contexts (e.g., Mosier and Skitka, 1999;Davis et al, 2020) and medical contexts (e.g., Goddard et al, 2012;Lyell et al, 2018), but have also been found in the military domain and in process control (Bahner et al, 2008;Parasuraman and Manzey, 2010) as well as in quality control (Kloker et al, 2022). However, automation bias can occur in every work field that includes human-systeminteraction (Goddard et al, 2012).…”

Section: Human Information Processing and Automation Biasmentioning

confidence: 99%

See 1 more Smart Citation

Check the box! How to deal with automation bias in AI-based personnel selection

et al. 2023

View full text Add to dashboard Cite

Artificial Intelligence (AI) as decision support for personnel preselection, e.g., in the form of a dashboard, promises a more effective and fairer selection process. However, AI-based decision support systems might prompt decision makers to thoughtlessly accept the system’s recommendation. As this so-called automation bias contradicts ethical and legal requirements of human oversight for the use of AI-based recommendations in personnel preselection, the present study investigates strategies to reduce automation bias and increase decision quality. Based on the Elaboration Likelihood Model, we assume that instructing decision makers about the possibility of system errors and their responsibility for the decision, as well as providing an appropriate level of data aggregation should encourage decision makers to process information systematically instead of heuristically. We conducted a 3 (general information, information about system errors, information about responsibility) x 2 (low vs. high aggregated data) experiment to investigate which strategy can reduce automation bias and enhance decision quality. We found that less automation bias in terms of higher scores on verification intensity indicators correlated with higher objective decision quality, i.e., more suitable applicants selected. Decision makers who received information about system errors scored higher on verification intensity indicators and rated subjective decision quality higher, but decision makers who were informed about their responsibility, unexpectedly, did not. Regarding aggregation level of data, decision makers of the highly aggregated data group spent less time on the level of the dashboard where highly aggregated data were presented. Our results show that it is important to inform decision makers who interact with AI-based decision-support systems about potential system errors and provide them with less aggregated data to reduce automation bias and enhance decision quality.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Human Information Processing and Automation Biasmentioning

confidence: 99%

Check the box! How to deal with automation bias in AI-based personnel selection

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Other literature has given more attention to the way acquiescence shows up as a learnt posture to the tedium of work, typically in the form of complacency or dulled senses. On airport security again, Davis et al (2020) interrogate how baggage screeners may be prone to exhibiting traits of 'automation bias' over time, as they overly depend on machines to make decisions for them. In a similar vein, but turning the gaze to the sky, Eastwood et al (2012) underscore the propensity for pilots to develop a sense of boredom when engaging in automatic flight, as the mind switches off and relinquishes control to aircraft avionics.…”

Section: Acquiescencementioning

confidence: 99%

Dispositions towards automation: Capital, technology, and labour relations in aeromobilities

Lin

Adey

Harris

2022

Dialogues in Human Geography

View full text Add to dashboard Cite

With the rapid rise of supercomputers, artificial intelligence, and advanced forms of robotics, recent years have seen a resurgence in interest in automation in the academy. In geography, scholars have yielded some important insights on the mottled relations between humans and machines as capital moves towards a more high-tech form of production. In this paper, we seek to extend these debates by delving into how labour's relations with automation do not always reactively vacillate between capitulation and adaptation. Instead, they are more ambiguously swayed by broader ‘dispositions towards automation’, which are generated either through practice, or by machines themselves that are designed to appeal to human desires and wants. Drawing on examples from aeromobilities and, particularly, airports, this paper shows how various in-process tendencies, or ‘points of meeting’, of human-technology assemblages enable and creatively rework automation in ways that are culturally specific and ideologically supportive of capital. We consider five different dispositions – namely, enchantment, aspiration, experimentation, gamification, and acquiescence – as starting points for further dialogues on technological relations and resistances in geography.

show abstract