Quantifying Hypothesis Space Misspecification in Learning From Human–Robot Demonstrations and Physical Corrections

Bobu, Andreea; Bajcsy, Andrea; Fisac, Jaime F.; Deglurkar, Sampada; Dragan, Anca D.

doi:10.1109/tro.2020.2971415

Cited by 36 publications

(47 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Typically, is fixed, recovering the Maximum Entropy IRL [30] observation model. Inspired by work in [5,12,13], we instead reinterpret it as a confidence in the robot's features' ability to explain human data. To detect missing features, we estimateˆvia a Bayesian belief update ′ ( , ) ∝ ( | , ) ( , ).…”

Section: Confidence Estimationmentioning

confidence: 99%

“…Unfortunately, this puts too much burden on system designers: specifying a priori an exhaustive set of all the features that end-users might care about is impossible for real-world tasks. While prior work has enabled robots to at least detect that the features it has access to cannot explain the human's input [5], it is still unclear how the robot might then construct a feature that can explain it. A natural answer is in deep IRL methods [11,20,29], which learn rewards defined directly on the high-dimensional raw state (or observation) space, thereby constructing features automatically.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Feature Expansive Reward Learning

Bobu

Wiggert

Tomlin

et al. 2021

Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

Self Cite

View full text Add to dashboard Cite

When a person is not satisfied with how a robot performs a task, they can intervene to correct it. Reward learning methods enable the robot to adapt its reward function online based on such human input, but they rely on handcrafted features. When the correction cannot be explained by these features, recent work in deep Inverse Reinforcement Learning (IRL) suggests that the robot could ask for task demonstrations and recover a reward defined over the raw state space. Our insight is that rather than implicitly learning about the missing feature(s) from demonstrations, the robot should instead ask for data that explicitly teaches it about what it is missing. We introduce a new type of human input in which the person guides the robot from states where the feature being taught is highly expressed to states where it is not. We propose an algorithm for learning the feature from the raw state space and integrating it into the reward function. By focusing the human input on the missing feature, our method decreases sample complexity and improves generalization of the learned reward over the above deep IRL baseline. We show this in experiments with a physical 7DOF robot manipulator, as well as in a user study conducted in a simulated environment.

show abstract

Section: Confidence Estimationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Feature Expansive Reward Learning

Bobu

Wiggert

Tomlin

et al. 2021

Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction

Self Cite

View full text Add to dashboard Cite

show abstract

“…slow down the motion), which is not a predefined preference that a robot would take into account. The robot needs to be aware of its lack of ability to explain the human's intention by having less confidence, and then the reason about how to behave to meet people's requirements, which is potentially solved in [162,163].…”

Section: Incremental Learning From Correctionmentioning

confidence: 99%

Review of the techniques used in motor‐cognitive human‐robot skill transfer

Guan

Wang

Yang

2021

Cognitive Comp and Systems

View full text Add to dashboard Cite

A conventional robot programming method extensively limits the reusability of skills in the developmental aspect. Engineers programme a robot in a targeted manner for the realisation of predefined skills. The low reusability of general-purpose robot skills is mainly reflected in inability in novel and complex scenarios. Skill transfer aims to transfer human skills to general-purpose manipulators or mobile robots to replicate human-like behaviours. Skill transfer methods that are commonly used at present, such as learning from demonstrated (LfD) or imitation learning, endow the robot with the expert's lowlevel motor and high-level decision-making ability, so that skills can be reproduced and generalised according to perceived context. The improvement of robot cognition usually relates to an improvement in the autonomous high-level decision-making ability. Based on the idea of establishing a generic or specialised robot skill library, robots are expected to autonomously reason about the needs for using skills and plan compound movements according to sensory input. In recent years, in this area, many successful studies have demonstrated their effectiveness. Herein, a detailed review is provided on the transferring techniques of skills, applications, advancements, and limitations, especially in the LfD. Future research directions are also suggested.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

“…The human provides a sequence of physical corrections to guide the robot toward their preferred objective, i.e., placing the bag on the green region while also avoiding the obstacles on the left, and holding the bag upright without squeezing or stretching it. [4]. These works assume that the human makes corrections based only on their objective, without considering the other corrections they have already made or are planning to provide.…”

Section: Introductionmentioning

confidence: 99%

“…Learning from Corrections (Online). Recent research recognizes that physical human corrections are often intentional, and therefore informative [1]- [4]. These works learn about the human's underlying objective in real-time by comparing the current correction to the robot's previous behavior.…”

Section: Introductionmentioning

confidence: 99%

Learning Human Objectives from Sequences of Physical Corrections

Canberk

Losey

et al. 2021

2021 IEEE International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

When personal, assistive, and interactive robots make mistakes, humans naturally and intuitively correct those mistakes through physical interaction. In simple situations, one correction is sufficient to convey what the human wants. But when humans are working with multiple robots or the robot is performing an intricate task often the human must make several corrections to fix the robot's behavior. Prior research assumes each of these physical corrections are independent events, and learns from them one-at-a-time. However, this misses out on crucial information: each of these interactions are interconnected, and may only make sense if viewed together. Alternatively, other work reasons over the final trajectory produced by all of the human's corrections. But this method must wait until the end of the task to learn from corrections, as opposed to inferring from the corrections in an online fashion. In this paper we formalize an approach for learning from sequences of physical corrections during the current task. To do this we introduce an auxiliary reward that captures the human's trade-off between making corrections which improve the robot's immediate reward and long-term performance. We evaluate the resulting algorithm in remote and in-person human-robot experiments, and compare to both independent and final baselines. Our results indicate that users are best able to convey their objective when the robot reasons over their sequence of corrections.

show abstract

Quantifying Hypothesis Space Misspecification in Learning From Human–Robot Demonstrations and Physical Corrections

Cited by 36 publications

References 33 publications

Feature Expansive Reward Learning

Feature Expansive Reward Learning

Review of the techniques used in motor‐cognitive human‐robot skill transfer

Learning Human Objectives from Sequences of Physical Corrections

Contact Info

Product

Resources

About