Asking for Help Using Inverse Semantics

Abstract

Abstract-Robots inevitably fail, often without the ability to recover autonomously. We demonstrate an approach for enabling a robot to recover from failures by communicating its need for specific help to a human partner using natural language. Our approach automatically detects failures, then generates targeted spoken-language requests for help such as "Please give me the white table leg that is on the black table." Once the human partner has repaired the failure condition, the system resumes full autonomy. We present a novel inverse semantics algorithm for generating effective help requests. In contrast to forward semantic models that interpret natural language in terms of robot actions and perception, our inverse semantics algorithm generates requests by emulating the human's ability to interpret a request using the Generalized Grounding Graph (G 3 ) framework. To assess the effectiveness of our approach, we present a corpusbased online evaluation, as well as an end-to-end user study, demonstrating that our approach increases the effectiveness of human interventions compared to static requests for help.

Integrating human observer inferences into robot motion planning

“…Exaggeration is one of the 12 principles of animation [51]. However, nowhere did we inform the robot of what exaggeration is and how it might be useful for legibility.…”

Embodied Collaborative Referring Expression Generation in Situated Human-Robot Interaction

“…The robot needs to be able to describe its internal representation of the shared environment so that the human understands what it is talking about. This is particularly important for establishing common ground and supporting successful interaction [1,28]. Toward this goal, this paper explores embodied collaborative referring expression generation which incorporates nonverbal modalities in the collaborative process of referential communication.…”

BWIBots: A platform for bridging the gap between AI and human–robot interaction research

“…Furthermore, such a task cannot be solved using static visual object recognition methods as detecting whether an object is full or empty may often require the robot to perform a certain action on it (e.g., lift the object to measure the force it exerts on the arm). In this section, the research contribution focuses on solving the symbol grounding problem (Harnad 1990), a longstanding challenge in AI, where language is grounded using the robot's perception and action (Tellex et al 2011;Matuszek et al 2012;Krishnamurthy and Kollar 2013;Perera and Allen 2013;Kollar, Krishnamurthy and Strimel 2013;Tellex et al 2014;Matuszek et al 2014;Parde et al 2015;Spranger and Steels 2015). To address this problem, we enable a robot to undergo two distinct developmental stages:…”