Disentangled Relational Representations for Explaining and Learning from Demonstration

Abstract

Learning from demonstration is an effective method for human users to instruct desired robot behaviour. However, for most non-trivial tasks of practical interest, efficient learning from demonstration depends crucially on inductive bias in the chosen structure for rewards/costs and policies. We address the case where this inductive bias comes from an exchange with a human user. We propose a method in which a learning agent utilizes the information bottleneck layer of a high-parameter variational neural model, with auxiliary loss terms, in order to ground abstract concepts such as spatial relations. The concepts are referred to in natural language instructions and are manifested in the high-dimensional sensory input stream the agent receives from the world. We evaluate the properties of the latent space of the learned model in a photorealistic synthetic environment and particularly focus on examining its usability for downstream tasks. Additionally, through a series of controlled table-top manipulation experiments, we demonstrate that the learned manifold can be used to ground demonstrations as symbolic plans, which can then be executed on a PR2 robot.

Robust and Interpretable Grounding of Spatial References with Relation Networks

“…Reasoning about spatial references has been explored in various contexts such as instruction following for 2D and 3D navigation (MacMahon et al, 2006;Vogel and Jurafsky, 2010;Chen and Mooney, 2011;Artzi and Zettlemoyer, 2013;Kim and Mooney, 2013;Andreas and Klein, 2015;Fried et al, 2018;Liu et al, 2019;Jain et al, 2019;Gaddy and Klein, 2019;Hristov et al, 2019;Chen et al, 2019) and situated dialog for robotic manipulation (Skubic et al, 2002;Kruijff et al, 2007;Kelleher and Costello, 2009;Landsiedel et al, 2017). Most of these approaches utilize supervised data, either in the form of policy demonstrations or target geometric representations.…”

“…'above', 'below') to perceptual processes like visual signals. While such early grounding efforts were limited by computational bottlenecks, several deep neural architectures have been recently proposed that jointly process text and visual input (Janner et al, 2017;Misra et al, 2017;Bisk et al, 2016;Liu et al, 2019;Jain et al, 2019;Gaddy and Klein, 2019; arXiv:2005.00696v1 [cs.CL] 2 May 2020 Hristov et al, 2019;Yu et al, 2018). While these approaches have made significant advances in improving the ability of agents at following spatial instructions, they are either not easily interpretable or require pre-specified parameterization to induce interpretable modules (Bisk et al, 2018).…”

DiversityGAN: Diversity-Aware Vehicle Motion Prediction via Latent Semantic Sampling

“…[18] has shown that there exist semantics in the latent space of generative adverserial networks (GANs), and [19] successfully decomposes the latent factor in a GAN into structured semantic parts. In addition to GANs, [20] has learned disentangled latent representations in a variational autoencoder (VAE) framework to ground spatial relations between objects. Unlike the information bottleneck motivation of [19], we use metric learning [8] to capture information such as maneuvers and interactions.…”