Levels of Automation in Human Factors Models for Automation Design: Why We Might Consider Throwing the Baby Out With the Bathwater

Abstract

This paper responds to Kaber's reflections on the empirical grounding and design utility of the Levels of Automation (LOA) framework. We discuss the suitability of the existing human performance data for supporting design decisions in complex work environments. We question why human factors design guidance seems wedded to a model of questionable predictive value. We challenge the belief that LOA frameworks offer useful input to the design and operation of highly automated systems. Finally, we seek to expand the design space for human-automation interaction beyond the familiar human factors constructs. Taken together, our positions paint LOA frameworks as abstractions suffering a crisis of confidence that Kaber's remedies cannot restore. Differing perspectives on common groundProfessor David B. Kaber's position paper invites a welcome exchange of ideas about a central construct in human factors engineering: Levels of Automation (LOA). We applaud JCEDM for airing a range of responses to his views in the same issue. This approach offers a promising contrast to slow motion, ping pong exchanges between journals with competing perspectives on human factors science.The authors collaborate on human-automation interaction research as both empiricists [Burns et al., 2008;Lau et al., 2008aLau et al., , 2008bLau, Jamieson & Skraaning, 2014, 2016a, 2016b and designers [Hurlen, Skraaning, Myers, Jamieson & Carlson, 2015; Jamieson, Hurlen & Skraning, 2014;Skraaning, Hurlen, LeDarz & Jamieson, 2016]. We adopt an inductive approach to research, building knowledge through prototyping and experimentation in complex work environments. We seek to support designers by aligning that knowledge with the richness of the design problem, and not primarily through models of questionable predictive value. This perspective has evolved through twenty years of realistic simulator studies of human-automation interaction on complex process control tasks; where human factors models have shown little utility in predicting performance outcomes (Skraaning & Jamieson, in preparation). It is from this perspective that we respond to Kaber's comments on the empirical basis for, and design relevance of, LOA models.We concur with much of Kaber's critical consideration of the empirical evidence for LOA predictions. We agree that human factors research should move away from constrained

Automation Stages & Levels, 20 Years After

“…Almost any meta-analysis will contain substantial variance in effect size between its set of studies, with trends and effects ranging from strong to weak or even reversed (Sebok & Wickens, 2017). Indeed it appears that some of the key trends underlying the three functions in Figure 2 are quite muted or absent in the more complex, less controlled environments of the more naturalistic studies (Jamieson & Skraaning, 2018). Such absence may be expected given the greater complexity of forces influencing performance and the less constrained strategies employed by workers.…”

Reflections on Commentaries on “Issues in Human–Automation Interaction Modeling”

“…I agree that most of these studies were conducted with laboratory-based simulations of different types of systems with varying user populations. Jamieson and Skraaning (2018) presented a reanalysis of studies included in the Onnasch et al (2014) meta-analysis, with a reduced scope including only those investigations of “complex work settings” or “expert operators.” They contended that this variation on the analysis demonstrated reduced consistency of human response trends across degrees of automation (DOAs; i.e., stages crossed with levels). Wickens (2018) concurred with their findings.…”

Qualitative and quantitative approach to assess the potential for automating administrative tasks in general practice

“…The assumption has been that healthcare occupations are less likely to be affected because they rely heavily on interpersonal interactions, but they also include tasks that require information acquisition, information analysis and decision making, all characteristics amenable to automation. 17 In primary care, automatable tasks include checking written documents for errors, reviewing and writing letters, creating referral letters, organising staff rotas, creating reports and maintaining records. Technologies such as natural language processing, voice recognition, text summarisation, robotic process automation and machine learning to support the manipulation of information are all potentially applicable.…”