Sorry Dave, I'm Afraid I Can't Do That: Explaining Unachievable Robot Tasks Using Natural Language

Raman, Vasumathi; Lignos, Constantine; Finucane, Cameron; Lee, Kenton C.T.; Marcus, Mitchell P.; Kress-Gazit, Hadas

doi:10.15607/rss.2013.ix.023

Cited by 49 publications

(35 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, when engineering robotics software, the missions are first expressed using natural-language requirements. These are then specified using domain-specific languages, many of which have been proposed over the last decades [12]- [14], [38]. These languages are often integrated with development environments that are used to generate code that can be executed within simulators or real robots [15]- [17].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Specification Patterns for Robotic Missions

Menghi

Tsigkanos

Ghezzi

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

Mobile and general-purpose robots increasingly support our everyday life, requiring dependable robotics control software. Creating such software mainly amounts to implementing their complex behaviors known as missions. Recognizing the need, a large number of domain-specific specification languages has been proposed. These, in addition to traditional logical languages, allow the use of formally specified missions for synthesis, verification, simulation, or guiding the implementation. For instance, the logical language LTL is commonly used by experts to specify missions, as an input for planners, which synthesize the behavior a robot should have. Unfortunately, domain-specific languages are usually tied to specific robot models, while logical languages such as LTL are difficult to use by non-experts.We present a catalog of 22 mission specification patterns for mobile robots, together with tooling for instantiating, composing, and compiling the patterns to create mission specifications. The patterns provide solutions for recurrent specification problems, each of which detailing the usage intent, known uses, relationships to other patterns, and-most importantly-a template mission specification in temporal logic. Our tooling produces specifications expressed in the LTL and CTL temporal logics to be used by planners, simulators, or model checkers. The patterns originate from 245 realistic textual mission requirements extracted from the robotics literature, and they are evaluated upon a total of 441 real-world mission requirements and 1251 mission specifications. Five of these reflect scenarios we defined with two well-known industrial partners developing human-size robots. We validated our patterns' correctness with simulators and two real robots.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Conceptually, defining a robotic mission entails two problems. First, ambiguities in mission requirements that prevent precise and unambiguous specifications must be resolved [38], [42], [43]. Consider the very simple mission requirement "the robot shall visit the kitchen and the office."…”

Section: Introductionmentioning

confidence: 99%

Specification Patterns for Robotic Missions

Menghi

Tsigkanos

Ghezzi

et al. 2021

IIEEE Trans. Software Eng.

View full text Add to dashboard Cite

show abstract

“…In this work, we present a corpus that is an order of magnitude larger, and apply the sequence-to-sequence framework that allows grounding to the full space of LTL formulae (instead of the GR(1) fragment). In an orthogonal direction, Raman et al [44] demonstrated the benefit of verifying LTL behavior by identifying unsatisfiable parts of LTL formulae and reporting which natural language commands resulted in these failures. Although verifiable behavior is a significant advantage of using LTL [23], we do not focus on verification in this work.…”

Section: Related Workmentioning

confidence: 99%

Sequence-to-Sequence Language Grounding of Non-Markovian Task Specifications

Gopalan

Arumugam

Wong

et al. 2018

Robotics: Science and Systems XIV

View full text Add to dashboard Cite

Abstract-Often times, natural language commands issued to robots not only specify a particular target configuration or goal state but also outline constraints on how the robot goes about its execution. That is, the path taken to achieving some goal state is given equal importance to the goal state itself. One example of this could be instructing a wheeled robot to "go to the living room but avoid the kitchen," in order to avoid scuffing the floor. This class of behaviors poses a serious obstacle to existing language understanding for robotics approaches that map to either action sequences or goal state representations. Due to the non-Markovian nature of the objective, approaches in the former category must map to potentially unbounded action sequences whereas approaches in the latter category would require folding the entirety of a robot's trajectory into a (traditionally Markovian) state representation, resulting in an intractable decision-making problem. To resolve this challenge, we use a recently introduced probabilistic variant of Linear Temporal Logic (LTL) as a goal specification language for a Markov Decision Process (MDP). While demonstrating that standard neural sequence-to-sequence learning models can successfully ground language to this semantic representation, we also provide analysis that highlights generalization to novel, unseen logical forms as an open problem for this class of model. We evaluate our system within two simulated robot domains as well as on a physical robot, demonstrating accurate language grounding alongside a significant expansion in the space of interpretable robot behaviors.

show abstract

“…To allow users who may be unfamiliar with LTL to define specifications, some approaches such as that of [18] include a parser that automatically translates English sentences into LTL formulas. Many applications distinguish two primary types of properties allowed in a specification -safety properties, which guarantee that "something bad never happens", and liveness conditions, which state that "something good (eventually) happens".…”

Section: B Linear Temporal Logicmentioning

confidence: 99%

Combining neural networks and tree search for task and motion planning in challenging environments

Paxton¹,

Raman²,

Hager

et al. 2017

2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

Self Cite

View full text Add to dashboard Cite

Abstract-We consider task and motion planning in complex dynamic environments for problems expressed in terms of a set of Linear Temporal Logic (LTL) constraints, and a reward function. We propose a methodology based on reinforcement learning that employs deep neural networks to learn low-level control policies as well as task-level option policies. A major challenge in this setting, both for neural network approaches and classical planning, is the need to explore future worlds of a complex and interactive environment. To this end, we integrate Monte Carlo Tree Search with hierarchical neural net control policies trained on expressive LTL specifications. This paper investigates the ability of neural networks to learn both LTL constraints and control policies in order to generate task plans in complex environments. We demonstrate our approach in a simulated autonomous driving setting, where a vehicle must drive down a road in traffic, avoid collisions, and navigate an intersection, all while obeying given rules of the road.

show abstract

Sorry Dave, I'm Afraid I Can't Do That: Explaining Unachievable Robot Tasks Using Natural Language

Cited by 49 publications

References 22 publications

Specification Patterns for Robotic Missions

Specification Patterns for Robotic Missions

Sequence-to-Sequence Language Grounding of Non-Markovian Task Specifications

Combining neural networks and tree search for task and motion planning in challenging environments

Contact Info

Product

Resources

About