One-Shot Learning of Multi-Step Tasks from Observation via Activity Localization in Auxiliary Video

Abstract

Due to burdensome data requirements, learning from demonstration often falls short of its promise to allow users to quickly and naturally program robots. Demonstrations are inherently ambiguous and incomplete, making correct generalization to unseen situations difficult without a large number of demonstrations in varying conditions. By contrast, humans are often able to learn complex tasks from a single demonstration (typically observations without action labels) by leveraging context learned over a lifetime. Inspired by this capability, our goal is to enable robots to perform one-shot learning of multi-step tasks from observation by leveraging auxiliary video data as context. Our primary contribution is a novel system that achieves this goal by: (1) using a single user-segmented demonstration to define the primitive actions that comprise a task, (2) localizing additional examples of these actions in unsegmented auxiliary videos via a metalearningbased approach, (3) using these additional examples to learn a reward function for each action, and (4) performing reinforcement learning on top of the inferred reward functions to learn action policies that can be combined to accomplish the task. We empirically demonstrate that a robot can learn multistep tasks more effectively when provided auxiliary video, and that performance greatly improves when localizing individual actions, compared to learning from unsegmented videos.

Learning Generalizable Robotic Reward Functions from “In-The-Wild” Human Videos

“…Simulation has also been leveraged as supervision to learn such representations [32] or to produce human data with domain randomization [3]. Finally, meta-learning [54] and subtask discovery [41,20] have also been explored as techniques for acquiring robot rewards or demos from human videos. In contrast to the majority of these works, which usually study a small set of human videos in a similar domain as the robot, we explicitly focus on leveraging "in-the-wild" human videos, specifically large and diverse sets of crowd-sourced videos from the real world from an existing dataset, which contains many different individuals, viewpoints, backgrounds, objects, and tasks.…”

Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration

“…While, third-person imitation learning uses date from other agents or viewpoints [27,35]. Recent methods for one-shot imitation learning [8,11,13,40,41,42] can translate a single demonstration to an executable pol- icy. The most similar to ours is NTP [41] that also learns long-horizon tasks.…”

Towards Robust One-shot Task Execution using Knowledge Graph Embeddings

“…Prior work in LfD has tackled the challenging problem of extracting task plans from a single end-user demonstration [10], [11], [12], [13], [14], [15], [16], [17], [18]. These approaches present intuitive ways for end-users to program complex robot behaviors using kinesthetic teaching [10], virtual reality [11], GUI programming [12], or direct demonstration [18].…”