DROID: Minimizing the Reality Gap using Single-Shot Human Demonstration

Abstract

Reinforcement learning (RL) has demonstrated great success in the past several years. However, most of the scenarios focus on simulated environments. One of the main challenges of transferring the policy learned in a simulated environment to real world, is the discrepancy between the dynamics of the two environments. In prior works, Domain Randomization (DR) has been used to address the reality gap for both robotic locomotion and manipulation tasks. In this paper, we propose Domain Randomization Optimization IDentification (DROID), a novel framework to exploit single-shot human demonstration for identifying the simulator's distribution of dynamics parameters, and apply it to training a policy on a door opening task. Our results show that the proposed framework can identify the difference in dynamics between the simulated and the real worlds, and thus improve policy transfer by optimizing the simulator's randomization ranges. We further illustrate that based on these same identified parameters, our method can generalize the learned policy to different but related tasks.

Sim-to-Real Transfer for Optical Tactile Sensing

“…By incorporating tactile information in the framework, we include additional 30 binary tactile values, { ĉi |i = 1, 2, ..., 30} in the observation space. Each of these values is converted from the tactile electrical signal into binary value using a threshold κ as follows: c) Reward Shaping: We provide a shaped reward function R at each timestep in simulation for improving the learning efficiency, which is similar to [6] and defined as follows:…”

“…While the sense of touch is highly integrated in human perception, the same cannot be said about robotic manipulation. Compared to the sense of touch, other sensory information such as vision and force/torque is more commonly used in robotics [3]- [6]. Vision can provide global information of the environment such as spatial distances as well as local features like shapes and colors without the need of contact.…”