Automation failures on tasks easily performed by operators undermines trust in automated aids

Madhavan, Poornima; Wiegmann, Douglas A.; Lacson, Frank C.

doi:10.1037/e577042012-018

Cited by 10 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Likelihood automation would provide feedback about its current performance such that it would keep the human informed about its current reliability, or the likelihood that the support being provided is useful. This type of information has been shown to lead to appropriate trust (Dzindolet et al, 2003;Madhavan et al, 2003).…”

Section: Prior Knowledge About Reliability -Perceived Reliabilitymentioning

confidence: 99%

Conceptual Model of Human-Automation Interaction

Sánchez

2009

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Why is it important to understand human behavior in automated environments? The performance of a humanautomation system is a product of the quality of the support provided by the automation and the manner in which that support is used by the human. Therefore, the solution to reaching an optimal level of system performance does not lie exclusively within possible improvements to technological components, but also by understanding the interaction of humans with automated agents. A conceptual model of human-automation interaction is presented in this paper. The model includes most of the well-established variable relationships to date, as well as a brief description of the literature that supports each relationship.

show abstract

Section: Prior Knowledge About Reliability -Perceived Reliabilitymentioning

confidence: 99%

Conceptual Model of Human-Automation Interaction

Sánchez

2009

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

show abstract

“…Such possibility would be valuable to examine from both performance and human trust and acceptance perspectives. Fourth, users judge automation that fails at subjectively easy tasks more harshly (Madhavan et al, 2003). Given the simplicity of the task, perhaps training failures were viewed more harshly than expected.…”

Section: Limitations and Future Directionsmentioning

confidence: 99%

Dancing With Algorithms: Interaction Creates Greater Preference and Trust in Machine-Learned Behavior

Gutzwiller

Reeder

2020

Hum Factors

View full text Add to dashboard Cite

Objective We examined a method of machine learning (ML) to evaluate its potential to develop more trustworthy control of unmanned vehicle area search behaviors. Background ML typically lacks interaction with the user. Novel interactive machine learning (IML) techniques incorporate user feedback, enabling observation of emerging ML behaviors, and human collaboration during ML of a task. This may enable trust and recognition of these algorithms. Method Participants judged and selected behaviors in a low and a high interaction condition (IML) over the course of behavior evolution using ML. User trust in the outputs, as well as preference, and ability to discriminate and recognize the behaviors were measured. Results Compared to noninteractive techniques, IML behaviors were more trusted and preferred, as well as recognizable, separate from non-IML behaviors, and approached similar performance as pure ML models. Conclusion IML shows promise for creating behaviors by involving the user; this is the first extension of this technique for vehicle behavior model development targeting user satisfaction and is unique in its multifaceted evaluation of how users perceived, trusted, and implemented these learned controllers. Application There are many contexts where the brittleness of ML cannot be trusted, but the advantage of ML over traditional programmed behaviors may be large, as in some military operations where they could be scaled. IML in this early form appears to generate satisfactory behaviors without sacrificing performance, use, or trust in the behavior, but more work is necessary.

show abstract

“…Automation has developed rapidly in response to the demands of higher labor costs and the desire for improved precision 54. Nevertheless, automated systems remain dependent upon human operators to monitor the integrity of the system and intervene when necessary to either prevent a system failure or to ensure that the system is operating optimally 55,56. The need for operator intervention is unpredictable in both its frequency and its intensity in automated systems, thereby relegating the operator to the role of a passive observer 57,58…”

Section: Traditional Cognitive Engineeringmentioning

confidence: 99%

Cognitive engineering

Wilson

Helton

Wiggins

2012

WIRES Cognitive Science

View full text Add to dashboard Cite

Cognitive engineering is the application of cognitive psychology and related disciplines to the design and operation of human-machine systems. Cognitive engineering combines both detailed and close study of the human worker in the actual work context and the study of the worker in more controlled environments. Cognitive engineering combines multiple methods and perspectives to achieve the goal of improved system performance. Given the origins of experimental psychology itself in issues regarding the design of human-machine systems, cognitive engineering is a core, or fundamental, discipline within academic psychology. WIREs Cogn Sci 2013, 4:17-31. doi: 10.1002/wcs.1204 CONFLICT OF INTEREST: The authors declare no conflict of interest. For further resources related to this article, please visit the WIREs website.

show abstract

Automation failures on tasks easily performed by operators undermines trust in automated aids

Cited by 10 publications

References 0 publications

Conceptual Model of Human-Automation Interaction

Conceptual Model of Human-Automation Interaction

Dancing With Algorithms: Interaction Creates Greater Preference and Trust in Machine-Learned Behavior

Cognitive engineering

Contact Info

Product

Resources

About