Toward Information Theoretic Human-Robot Dialog

Tellex, Stefanie; Thaker, Pratiksha; Deits, Robin; Kollar, Thomas; Roy, Nicholas

doi:10.15607/rss.2012.viii.052

Cited by 41 publications

(39 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, we show that our entropy-based metric for identifying uncertain variables to ask questions about significantly reduces the number of questions the robot needs to ask in order to resolve its uncertainty. This work expands on previous work presented in (Simeonov, Tellex, Kollar, & Roy, 2011) and (Tellex, Thaker, Deits, Kollar, & Roy, 2012) with the introduction and evaluation of two new types of questions (yes-or-no and reset, described in Section 3.1) and a new metric to select questions that will most effectively reduce the robot's uncertainty about its inferred sequence of actions (Metric 3 [Event Entropy], introduced in Section 3.1.2).…”

Section: Introductionsupporting

confidence: 87%

Clarifying Commands with Information-Theoretic Human-Robot Dialog

Deits

Tellex²,

Thaker³

et al. 2013

Journal of Human-Robot Interaction

Self Cite

View full text Add to dashboard Cite

Our goal is to improve the efficiency and effectiveness of natural language communication between humans and robots. Human language is frequently ambiguous, and a robot's limited sensing makes complete understanding of a statement even more difficult. To address these challenges, we describe an approach for enabling a robot to engage in clarifying dialog with a human partner, just as a human might do in a similar situation. Given an unconstrained command from a human operator, the robot asks one or more questions and receives natural language answers from the human. We apply an information-theoretic approach to choosing questions for the robot to ask. Specifically, we choose the type and subject of questions in order to maximize the reduction in Shannon entropy of the robot's mapping between language and entities in the world. Within the framework of the G 3 graphical model, we derive a method to estimate this entropy reduction, choose the optimal question to ask, and merge the information gained from the human operator's answer. We demonstrate that this improves the accuracy of command understanding over prior work while asking fewer questions as compared to baseline question-selection strategies.

show abstract

Section: Introductionsupporting

confidence: 87%

Clarifying Commands with Information-Theoretic Human-Robot Dialog

Deits

Tellex²,

Thaker³

et al. 2013

Journal of Human-Robot Interaction

Self Cite

View full text Add to dashboard Cite

show abstract

“…In such work, clarification requests have taken the form of yes/no questions about the properties of an object [5,9,14] or generic wh-questions (e.g., "What do the words X refer to?") [22,14].…”

Section: A Previous Workmentioning

confidence: 99%

Resolution of Referential Ambiguity in Human-Robot Dialogue Using Dempster-Shafer Theoretic Pragmatics

Williams¹,

Scheutz²

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-Robots designed to interact with humans in realistic environments must be able to handle uncertainty with respect to the identities and properties of the people, places, and things found in their environments. When humans refer to these entities using under-specified language, robots must often generate clarification requests to determine which entities were meant. In this paper, we present recommendations for designers of robots needing to generate such requests, and show how a Dempster-Shafer theoretic pragmatic reasoning component capable of generating requests to clarify pragmatic uncertainty can also generate requests to resolve referential uncertainty when integrated with a probabilistic reference resolution component.

show abstract

“…One class of solutions [41,28,8,5,31,30] considers the problem as one of parsing free-form commands into their formal language equivalent, which a planner takes as input. Other approaches [20,43,44] function by mapping free-form utterances into their corresponding object and action referents in the robot's world model. With the exception of MacMahon et al [28] and Matuszek et al [30], existing methods assume the map is known a priori.…”

Section: Related Workmentioning

confidence: 99%

A framework for learning semantic maps from grounded natural language descriptions

Walter

Hemachandra

Homberg

et al. 2014

The International Journal of Robotics Research

Self Cite

View full text Add to dashboard Cite

This paper describes a framework that enables robots to efficiently learn human-centric models of their environment from natural language descriptions. Typical semantic mapping approaches are limited to augmenting metric maps with higherlevel properties of the robot's surroundings (e.g., place type, object locations) that can be inferred from the robot's sensor data, but do not use this information to improve the metric map. The novelty of our algorithm lies in fusing high-level knowledge that people can uniquely provide through speech with metric information from the robot's low-level sensor streams. Our method jointly estimates a hybrid metric, topological, and semantic representation of the environment. This semantic graph provides a common framework in which we integrate information that the user communicates (e.g., labels and spatial relations) with metric observations from low-level sensors. Our algorithm efficiently maintains a factored distribution over semantic graphs based upon the stream of natural language and low-level sensor information. We detail the means by which the framework incorporates knowledge conveyed by the user's descriptions, including the ability to reason over expressions that reference yet unknown regions in the environment. We evaluate the algorithm's ability to learn human-centric maps of several different environments and analyze the knowledge inferred from language and the utility of the learned maps. The results demonstrate that the incorporation of information from free-form descriptions increases the metric, topological and semantic accuracy of the recovered environment model.

show abstract

Toward Information Theoretic Human-Robot Dialog

Cited by 41 publications

References 19 publications

Clarifying Commands with Information-Theoretic Human-Robot Dialog

Clarifying Commands with Information-Theoretic Human-Robot Dialog

Resolution of Referential Ambiguity in Human-Robot Dialogue Using Dempster-Shafer Theoretic Pragmatics

A framework for learning semantic maps from grounded natural language descriptions

Contact Info

Product

Resources

About