Multimodal real-time contingency detection for HRI

Chu, Vivian; Bullard, Kalesha; Thomaz, Andrea L.

doi:10.1109/iros.2014.6943025

Cited by 9 publications

(4 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Existing work has also implicitly modeled a person's availability through methods termed "contingency detection" [14,22,60]. These methods often assume the person as available, perform a probe action (or sequence of actions), and then reassess the person's availability based on the person's response.…”

Section: Estimating Availability and Interruption Contextmentioning

confidence: 99%

Robot Classification of Human Interruptibility and a Study of Its Effects

Banerjee

Silva

Chernova

2018

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

As robots become increasingly prevalent in human environments, there will inevitably be times when the robot needs to interrupt a human to initiate an interaction. Our work introduces the first interruptibility-aware mobile-robot system, which uses social and contextual cues online to accurately determine when to interrupt a person. We evaluate multiple non-temporal and temporal models on the interruptibility classification task, and show that a variant of Conditional Random Fields (CRFs), the Latent-Dynamic CRF, is the most robust, accurate, and appropriate model for use on our system. Additionally, we evaluate different classification features and show that the observed demeanor of a person can help in interruptibility classification; but in the presence of detection noise, robust detection of object labels as a visual cue to the interruption context can improve interruptibility estimates. Finally, we deploy our system in a large-scale user study to understand the effects of interruptibility-awareness on human-task performance, robot-task performance, and on human interpretation of the robot's social aptitude. Our results show that while participants are able to maintain task performance, even in the presence of interruptions, interruptibility-awareness improves the robot's task performance and improves participant social perceptions of the robot.

show abstract

Section: Estimating Availability and Interruption Contextmentioning

confidence: 99%

Robot Classification of Human Interruptibility and a Study of Its Effects

Banerjee

Silva

Chernova

2018

J. Hum.-Robot Interact.

View full text Add to dashboard Cite

show abstract

“…Movellan (2005) implemented a contingency detection controller using binary audio signals; the agent’s action timing used an information-maximizing approach inspired by infant vocalizations. Contingency detection has also been implemented by Lee et al (2011) and Chu et al (2014) using vision-based approaches. In turn-taking, the ongoing perception and actuation of contingent events are the glue that binds together the entire reciprocal interaction and gives it its cyclic rhythm.…”

Section: Related Workmentioning

confidence: 99%

“…timing modeling and management (Broz, 2008; Chu et al, 2014; Lee et al, 2011; Movellan, 2005), so that interaction dynamics meet natural human expectations about when actions should be taken.…”

Section: Approachmentioning

confidence: 99%

“…• concurrency of actions across participants working towards a shared goal (Hoffman and Breazeal, 2007;Shah et al, 2011), as well as of actions across the communicative modalities of a single participant Huang and Mutlu, 2014;Kopp et al, 2006;Nooraei et al, 2014); • sequences of conditions that must be met, such as executing all of the phases of a gestural action (Salem et al, 2012) or dispatching events when following a plan (Shah et al, 2011); • timing modeling and management (Broz, 2008;Chu et al, 2014;Lee et al, 2011;Movellan, 2005), so that interaction dynamics meet natural human expectations about when actions should be taken.…”

Section: Requirements Of Social Interactionmentioning

confidence: 99%

See 1 more Smart Citation

Timed Petri nets for fluent turn-taking over multimodal interaction resources in human-robot collaboration

Chao

Thomaz

2016

The International Journal of Robotics Research

View full text Add to dashboard Cite

The goal of this work is to develop computational models of social intelligence that enable robots to work side by side with humans, solving problems and achieving task goals through dialogue and collaborative manipulation. A defining problem of collaborative behavior in an embodied setting is the manner in which multiple agents make use of shared resources. In a situated dialogue, these resources include physical bottlenecks such as objects or spatial regions, and cognitive bottlenecks such as the speaking floor. For a robot to function as an effective collaborative partner with a human, it must be able to seize and yield such resources appropriately according to social expectations. We describe a general framework that uses timed Petri nets for the modeling and execution of robot speech, gaze, gesture, and manipulation for collaboration. The system dynamically monitors resource requirements and availability to control real-time turn-taking decisions over resources that are shared with humans, reasoning about different resource types independently. We evaluate our approach with an experiment in which our robot Simon performs a collaborative assembly task with 26 different human partners, showing that the multimodal reciprocal approach results in superior task performance, fluency, and balance of control.

show abstract