Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning

Abstract

Dexterous manipulation of arbitrary objects, a fundamental daily task for humans, has been a grand challenge for autonomous robotic systems. Although data-driven approaches using reinforcement learning can develop specialist policies that discover behaviors to control a single object, they often exhibit poor generalization to unseen ones. In this work, we show that policies learned by existing reinforcement learning algorithms can in fact be generalist when combined with multi-task learning and a well-chosen object representation. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse realworld objects and generalize to new objects with unseen shape or size. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms the single-object specialist policies on both training as well as held-out test objects. Video results at https://huangwl18.github.io/geometry-dex.• We show that a multi-task policy trained on many objects can match the performance of the oracles, i.e. policies trained on those objects individually. • We present a simple representation encoder that not only

Masked Visual Pre-training for Motor Control

“…Recently, OpenAI (OpenAI et al, 2020; has shown impressive results in dexterous in-hand manipulation with large scale domain randomization. A number of works consider related manipulation problems with multi-finger hands but rely on explicit state estimation (Handa et al, 2020;Huang et al, 2021), expert policies (Chen et al, 2021a), human demonstrations (Rajeswaran et al, 2018;Radosavovic et al, 2021;Qin et al, 2021), human priors (Mandikal & Grauman, 2021), or models (Nagabandi et al, 2019). In contrast, we do not use on any of the aforementioned components in our approach.…”

Learning Generalizable Dexterous Manipulation from Human Grasp Affordance

“…But training with RL still suffers from high sample complexity and it might also need unexpected and unsafe behaviors given the high-dimensional action and state space. Extending from this line of research, recent efforts [11,24] have been made on using model-free RL to generalize diverse object in-hand reorientation. While these works have shown encouraging results, it is unclear how well it works for grasping and relocating objects, which is the main focus for this paper.…”

“…Our policy takes both the point cloud of the object, object 6D pose, robot hand joint states as inputs, and predicts the actions for the robot hand. Specifically, to represent the object shape, we utilize the PointNet [24] encoder θ pc for the point cloud inputs. Given the point cloud embedding, we concatenate it with the object 6D pose parameters and hand joint states together and forward them together to a 3layer MLP θ p network for decision making.…”

Abstract representations emerge naturally in neural networks trained to perform multiple tasks

“…To investigate this hypothesis, we train feedforward neural network models to perform multiple distinct tasks on a common stimulus space. Previous work in machine learning has shown that similar multi-tasking networks can achieve lower loss from the same number of samples than networks trained independently on each task[25] (and see [26]), and that they can quickly learn novel, but related, tasks that are introduced after training[27]. Both of these properties are hallmarks of abstract representations – however, to our knowledge, the representational geometry developed by these multi-tasking networks has not been characterized.…”