Enabling Situation Awareness Under High Levels of Automation: Results From An Experimental Study

Abstract

Automation poses a great opportunity to increase both system and human performance. However, it is important to pay attention to the design of the collaboration between the human operator and the automated systems. If not designed correctly, the overall performance may be severely reduced. Previous research indicates that an intermediate level of automation might be the solution that minimizes the tradeoffs between manual and automated actions. However, there are still many remaining unanswered questions related to impacts on situation awareness (SA) and workload during different levels of automation (LOAs). Some of the documented issues of automation include reduced SA, complacency, and over reliance on the system automation.The Advanced Reactor Technologies Human Automation Collaboration Research Project has conducted two experimental studies investigating the effect of different LOAs on system and human performance. The results from the first experimental study were published in Oxstrand and Le Blanc (2014) and Le Blanc and Oxstrand (2015) and are summarized in this report. The second experimental study was conducted by Idaho National Laboratory researchers. This report focuses on the second experimental study and details the design of the study and results.The second experimental study aimed to investigate the effects automation has on overall human-system performance. Researchers employed a simplified process control simulation using a nested experimental design to evaluate performance differences between four LOAs (manual, intermediate, adaptable, and automatic) with varying faults and conditional changes occurring during each scenario. The effects of automation were measured by participants' ability to detect and respond to failing automation and conditional changes using a variety of metrics assessing SA, system performance, and workload. The intent of this study is to determine the optimum conditions for human automation collaboration.The results of the present study indicate that more automation led to better system performance under normal operating conditions, and poorer SA and system performance under fault conditions. The results for the two middle LOAs are not as straightforward. Consistent with the hypothesis, system performance using adaptable automation exceeded system performance when using the intermediate LOA. However, contrary to the hypotheses, SA and fault performance were not superior using the adaptable LOA than using the intermediate LOA. The results of this study indicate that although adaptable automation fulfills half if its promise (i.e., it produces system performance closer to fully automatic than intermediate LOAs), it falls short of also enhancing SA and fault performance. This study demonstrates the fundamental tradeoff associated with high LOAs: automation enhances performance during normal conditions, but increases failure (sometimes catastrophically) of the humansystem under the condition of automation failure. In contrast, although manual performance is inferior to automati...

Digitalization Guiding Principles and Method for Nuclear Industry Work Processes

“…McLeod (2022) recommends that humans must remain proactive in their engaging behaviors with the system as opposed to reactive to combat dissociation and withdrawal and to maintain awareness. HFE researchers at INL have studied situation awareness within the context of nuclear operations extensively (Le Blanc et al, 2015;Oxstrand & Le Blanc, 2014;Spielman et al, 2017).…”