Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

Rajeswaran, Aravind; Kumar, Vikas; Gupta, Abhishek; Vezzani, Giulia; John, Sabu; Todorov, Emanuel; Levine, Sergey

doi:10.48550/arxiv.1709.10087

Cited by 139 publications

(221 citation statements)

References 0 publications

Supporting

Mentioning

220

Contrasting

Order By: Relevance

“…Data driven models are observed to be utilized frequently for speculating the behavior of complex systems in various domains such as biological [1], mechanical [2], [3], robotics [4]- [6], and energy forecasting [7] systems. In certain cases, wherein a "full physics" (high fidelity) model is unavailable or is computationally prohibitive to evaluate, a "partial physics" (lower fidelity) model can be used for prediction.…”

Section: A Physics-infused Hybrid Modelingmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Iqbal¹,

Behjat²,

Adlakha³

et al. 2022

Preprint

View full text Add to dashboard Cite

Physics-Infused Machine Learning (PIML) architectures aim at integrating machine learning with computationallyefficient, low-fidelity (partial) physics models, leading to improved generalizability, extrapolability, and robustness to noise, compared to pure data-driven approximation models. End-uses of PIML include, but are not limited to, model-based optimization and model-predictive control. Recently a new PIML architecture was reported by the same authors, known as Opportunistic Physics-mining Transfer Mapping Architecture or OPTMA, which transfers the original inputs into latent features using a transfer neural network; the partial physics model then uses the latent features to generate the final output that is as close as possible to the high-fidelity output. While gradient-free solvers and back-propagation with supervised learning (where optimum labels are pre-generated) have been used to train OPTMA, that approach remains computationally inefficient, and challenging to generalize across different problems or exploit state-of-theart machine learning architectures. This paper aims to alleviate these issues by infusing the partial physics model inside the neural network, as described via tensors in the popular machine learning framework, PyTorch. Such a description also naturally allows auto-differentiation of the partial physics model, thereby enabling the use of efficient back-propagation methods to train the transfer network. The benefits of the upgraded OPTMA architecture with Automatic Differentiation (OPTMA-Net) is demonstrated by applying it to the problem of modeling the sound pressure field created by a hovering unmanned aerial vehicle (UAV). Ground truth data for this problem has been obtained from an indoor UAV noise measurement setup. Here, the partial physics model is based on the interference of acoustic pressure waves generated by an arbitrary number of acoustic monopole sources. Case studies show that OPTMA-Net provides generalization performance close to, and extrapolation performance that is 4 times better than, those given by a pure data-driven model.

show abstract

Section: A Physics-infused Hybrid Modelingmentioning

confidence: 99%

“…An ideal transfer model maps the transferred feature for the partial physics model such that the partial physics model generates the actual outputs (i.e., the full physics experimental values). Equation (6) show the role of transfer model for the partial physics model.…”

Section: B Partial Physics Acoustic Modelmentioning

confidence: 99%

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Iqbal¹,

Behjat²,

Adlakha³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Van Hoof et al [40] learned the tactile informed policy via RL for a three-finger manipulator to move an object on the table. To reduce the sample complexity of model-free learning, [9,13,8,41,6] combined imitation learning with RL to learn to reorient a pen, open a door, assemble LEGO blocks, etc. However, collecting expert demonstration data from humans is expensive, time-consuming, and even incredibly difficult for contact-rick tasks [8].…”

Section: Related Workmentioning

confidence: 99%

“…As such, it remains unclear how to scale these methods to real-world and generalize to new objects. Another line of work has employed (ii) model-based reinforcement learning [4,5]; or (iii) model-free reinforcement learning with [6,7,8,9] and without expert demonstrations [10,11,12,13]. While some of these works demonstrated learned skills on real robots, it required use of additional sensory apparatus not readily available in the real-world (e.g., motion capture system) to infer the object state, and the learned policies did not generalize to diverse objects.…”

Section: Introductionmentioning

confidence: 99%

A System for General In-Hand Object Re-Orientation

Chen¹,

Xu²,

Agrawal³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Most robot manipulation simulations often concentrate on solving specific manipulation problems, such as grasping [17,18] and in-hand manipulation [19][20][21]. The Recent MetaWorld [22] and RLBench [23] contain a variety of manipulation tasks for robot learning.…”

Section: A Simulation Environments For Robotic Manipulationmentioning

confidence: 99%

RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment

Yu¹,

Shen²,

Chen³

et al. 2021

Preprint

View full text Add to dashboard Cite

Part assembly is a typical but challenging task in robotics, where robots assemble a set of individual parts into a complete shape. In this paper, we develop a robotic assembly simulation environment for furniture assembly. We formulate the part assembly task as a concrete reinforcement learning problem and propose a pipeline for robots to learn to assemble a diverse set of chairs. Experiments show that when testing with unseen chairs, our approach achieves a success rate of 74.5% under the object-centric setting and 50.0% under the full setting. We adopt an RRT-Connect [1] algorithm as the baseline, which only achieves a success rate of 18.8% after a significantly longer computation time. Supplemental materials and videos are available on our project webpage: https://sites.google.com/view/roboticassembly.

show abstract

Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations

Cited by 139 publications

References 0 publications

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

Efficient Training of Transfer Mapping in Physics-Infused Machine Learning Models of UAV Acoustic Field

A System for General In-Hand Object Re-Orientation

RoboAssembly: Learning Generalizable Furniture Assembly Policy in a Novel Multi-robot Contact-rich Simulation Environment

Contact Info

Product

Resources

About