Head and shoulders: automatic error detection in human-robot interaction

Trung, Pauline; Giuliani, Manuel; Miksch, Michael; Stollnberger, Gerald; Stadler, Susanne; Mirnig, Nicole; Tscheligi, Manfred

doi:10.1145/3136755.3136785

Cited by 22 publications

(13 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research in HRI has also investigated how robot failures impact user behaviours, including patterns in eye-gaze, head movements, and speech -social signals that exhibit either established grounding sequences or implicit behavioural responses to failures [6,31,35,76,80]. Behavioural signals have also been examined at unexpected responses from human-robot interactions in the wild [5,30,75], with the use of social signals from low-level sensor input, to highlevel features that represent affect, attention and engagement.…”

Section: Robot Failuresmentioning

confidence: 99%

Grounding behaviours with conversational interfaces: effects of embodiment and failures

Kontogiorgos

Pereira

Gustafson

2021

J Multimodal User Interfaces

View full text Add to dashboard Cite

Conversational interfaces that interact with humans need to continuously establish, maintain and repair common ground in task-oriented dialogues. Uncertainty, repairs and acknowledgements are expressed in user behaviour in the continuous efforts of the conversational partners to maintain mutual understanding. Users change their behaviour when interacting with systems in different forms of embodiment, which affects the abilities of these interfaces to observe users’ recurrent social signals. Additionally, humans are intellectually biased towards social activity when facing anthropomorphic agents or when presented with subtle social cues. Two studies are presented in this paper examining how humans interact in a referential communication task with wizarded interfaces in different forms of embodiment. In study 1 (N = 30), we test whether humans respond the same way to agents, in different forms of embodiment and social behaviour. In study 2 (N = 44), we replicate the same task and agents but introduce conversational failures disrupting the process of grounding. Findings indicate that it is not always favourable for agents to be anthropomorphised or to communicate with non-verbal cues, as human grounding behaviours change when embodiment and failures are manipulated.

show abstract

Section: Robot Failuresmentioning

confidence: 99%

Grounding behaviours with conversational interfaces: effects of embodiment and failures

Kontogiorgos

Pereira

Gustafson

2021

J Multimodal User Interfaces

View full text Add to dashboard Cite

show abstract

“…People seem to have various predictable behavioral responses to robotic failures that can be used by robots to identify when a failure has occurred. Failure has been shown to influence users' gaze patterns (Gehle et al, 2015 ; Hayes et al, 2016 ; Mirnig et al, 2017 ), facial expressions (Hayes et al, 2016 ; Mirnig et al, 2017 ), head movements (Hayes et al, 2016 ; Mirnig et al, 2017 ; Trung et al, 2017 ), body movements (Mirnig et al, 2017 ; Trung et al, 2017 ), and verbal communication (Gieselmann, 2006 ; Giuliani et al, 2015 ). Gieselmann ( 2006 ) found that indicators for errors in human-robot conversation included sudden changes of the current dialogue topic, indicating non-understanding by asking unspecific questions, asking for additional information and repeating the previous question.…”

Section: A Unified Information Processing Model For User Centered Faimentioning

confidence: 99%

Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development

2018

View full text Add to dashboard Cite

While substantial effort has been invested in making robots more reliable, experience demonstrates that robots operating in unstructured environments are often challenged by frequent failures. Despite this, robots have not yet reached a level of design that allows effective management of faulty or unexpected behavior by untrained users. To understand why this may be the case, an in-depth literature review was done to explore when people perceive and resolve robot failures, how robots communicate failure, how failures influence people's perceptions and feelings toward robots, and how these effects can be mitigated. Fifty-two studies were identified relating to communicating failures and their causes, the influence of failures on human-robot interaction (HRI), and mitigating failures. Since little research has been done on these topics within the HRI community, insights from the fields of human computer interaction (HCI), human factors engineering, cognitive engineering and experimental psychology are presented and discussed. Based on the literature, we developed a model of information processing for robotic failures (Robot Failure Human Information Processing, RF-HIP), that guides the discussion of our findings. The model describes the way people perceive, process, and act on failures in human robot interaction. The model includes three main parts: (1) communicating failures, (2) perception and comprehension of failures, and (3) solving failures. Each part contains several stages, all influenced by contextual considerations and mitigation strategies. Several gaps in the literature have become evident as a result of this evaluation. More focus has been given to technical failures than interaction failures. Few studies focused on human errors, on communicating failures, or the cognitive, psychological, and social determinants that impact the design of mitigation strategies. By providing the stages of human information processing, RF-HIP can be used as a tool to promote the development of user-centered failure-handling strategies for HRIs.

show abstract

“…In addition, people respond differently when facing social norm violations and technical failures; in particular, technical failures generally resulted in fewer social signals and faster reaction times [20]. Prior works have demonstrated how social signals such as upper body movements (e.g., [28]), gaze (e.g., [2,3]), and gestures (e.g., [3]) can be used to detect errors effectively. It is worth noting that most of these prior works used human-like robots, which have been shown to elicit different responses to failures than non-humanoid embodiments in social error scenarios [17].…”

Section: Background and Related Workmentioning

confidence: 99%

“…To identify robot errors during situated interactions with people, prior research has investigated how people may respond to a robot's technical and social errors (e.g., [10,13]) and explored how social signals may be used for error detection [28]. To date, research has mostly used anthropomorphic robots when studying human responses to robot errors; not much has explored how people would exhibit social signals in response to errors produced by non-anthropomorphic robots (e.g., manipulators) that are commonly deployed in workplaces, such as factories.…”

Section: Introductionmentioning

confidence: 99%

Not All Errors Are Created Equal: Exploring Human Responses to Robot Errors with Varying Severity

Stiber

Huang

2020

Companion Publication of the 2020 International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Head and shoulders: automatic error detection in human-robot interaction

Cited by 22 publications

References 10 publications

Grounding behaviours with conversational interfaces: effects of embodiment and failures

Grounding behaviours with conversational interfaces: effects of embodiment and failures

Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development

Not All Errors Are Created Equal: Exploring Human Responses to Robot Errors with Varying Severity

Contact Info

Product

Resources

About