Elaborating on Learned Demonstrations with Temporal Logic Specifications

Abstract

Most current methods for learning from demonstrations assume that those demonstrations alone are sufficient to learn the underlying task. This is often untrue, especially if extra safety specifications exist which were not present in the original demonstrations. In this paper, we allow an expert to elaborate on their original demonstration with additional specification information using linear temporal logic (LTL). Our system converts LTL specifications into a differentiable loss. This loss is then used to learn a dynamic movement primitive that satisfies the underlying specification, while remaining close to the original demonstration. Further, by leveraging adversarial training, the system learns to robustly satisfy the given LTL specification on unseen inputs, not just those seen in training. We show that our method is expressive enough to work across a variety of common movement specification patterns such as obstacle avoidance, patrolling, keeping steady, and speed limitation. In addition, we show how to modify a base demonstration with complex specifications by incrementally composing multiple simpler specifications. We also implement our system on a PR-2 robot to show how a demonstrator can start with an initial (sub-optimal) demonstration, then interactively improve task success by including additional specifications enforced with a differentiable LTL loss.

Temporal Logic Guided Motion Primitives for Complex Manipulation Tasks with User Preferences

“…Compared with most existing works, the use of TLTL not only enables the encoding of complex tasks that involve a sequence of logically organized action plans, but also provides a convenient and effective means to design the cost function. Close to our work, LTL specifications was also incorporated in the learning of DMPs in [32]. However, the loss function designed in [32] is limited in evaluating whether or not a given LTL specification is satisfied and the log-sum-exponential approximation is over-approximated.…”

“…Close to our work, LTL specifications was also incorporated in the learning of DMPs in [32]. However, the loss function designed in [32] is limited in evaluating whether or not a given LTL specification is satisfied and the log-sum-exponential approximation is over-approximated. In contrast, the weighted TLTL robustness in this work is sound that can not only qualitatively evaluate the satisfaction of LTL specifications, but also quantitively determine its satisfaction degree.…”

Action sequencing using visual permutations

“…Existing attempts to extend neural models to handle these types of constraints are often made in a post-hoc fashion (eg. action clipping [14], elaboration using auxiliary losses [15]).…”

Semi-Supervised Trajectory-Feedback Controller Synthesis for Signal Temporal Logic Specifications

“…In contrast to LTL which is defined over atomic propositions (i.e., discrete states), STL is defined over continuous real-valued signals and encompasses a notion of robustness, a scalar measuring the degree of specification satisfaction/violation. Accordingly, there has been a growing interest in using STL robustness in gradient-based methods for controller synthesis (e.g., [7], [8], [9], [10]). Recently, stlcg [11], a toolbox leveraging Pytorch [12] to compute STL robustness, was developed.…”