BayesSim: Adaptive Domain Randomization Via Probabilistic Inference for Robotics Simulators

Ramos, Fábio; Possas, Rafael; Fox, Dieter

doi:10.15607/rss.2019.xv.029

Cited by 89 publications

(65 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though as the possible dimensions of the randomization increases, its scalability issues are becoming serious [130]. And on the other hand, regarding [131], too many randomizations imply a conservative policy from the agent. Although most research presented use some randomization (multiple tracks, random initialization or goal, etc.…”

Section: B Sim2realmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

343

View full text Add to dashboard Cite

Academic research in the field of autonomous vehicles has reached high popularity in recent years related to several topics as sensor technologies, V2X communications, safety, security, decision making, control, and even legal and standardization rules. Besides classic control design approaches, Artificial Intelligence and Machine Learning methods are present in almost all of these fields. Another part of research focuses on different layers of Motion Planning, such as strategic decisions, trajectory planning, and control. A wide range of techniques in Machine Learning itself have been developed, and this article describes one of these fields, Deep Reinforcement Learning (DRL). The paper provides insight into the hierarchical motion planning problem and describes the basics of DRL. The main elements of designing such a system are the modeling of the environment, the modeling abstractions, the description of the state and the perception models, the appropriate rewarding, and the realization of the underlying neural network. The paper describes vehicle models, simulation possibilities and computational requirements. Strategic decisions on different layers and the observation models, e.g., continuous and discrete state representations, grid-based, and camera-based solutions are presented. The paper surveys the state-of-art solutions systematized by the different tasks and levels of autonomous driving, such as carfollowing, lane-keeping, trajectory following, merging, or driving in dense traffic. Finally, open questions and future challenges are discussed.

show abstract

Section: B Sim2realmentioning

confidence: 99%

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Aradi

2022

IEEE Trans. Intell. Transport. Syst.

343

View full text Add to dashboard Cite

show abstract

“…Probabilistic inference techniques, on the other hand, seek to infer a distribution of simulation parameters that allows downstream applications to evaluate the uncertainty of the estimates. Such methods have been applied to learn conditional densities of simulation parameters given trajectories from the simulator and the real system [27,52,53,68,73,80].…”

Section: Parameter Inference For Simulatorsmentioning

confidence: 99%

DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting

Heiden

Macklin

Narang

et al. 2021

Robotics: Science and Systems XVII

Self Cite

View full text Add to dashboard Cite

Robotic cutting of soft materials is critical for applications such as food processing, household automation, and surgical manipulation. As in other areas of robotics, simulators can facilitate controller verification, policy learning, and dataset generation. Moreover, differentiable simulators can enable gradient-based optimization, which is invaluable for calibrating simulation parameters and optimizing controllers. In this work, we present DiSECt: the first differentiable simulator for cutting soft materials. The simulator augments the finite element method (FEM) with a continuous contact model based on signed distance fields (SDF), as well as a continuous damage model that inserts springs on opposite sides of the cutting plane and allows them to weaken until zero stiffness, enabling crack formation. Through various experiments, we evaluate the performance of the simulator. We first show that the simulator can be calibrated to match resultant forces and deformation fields from a state-of-the-art commercial solver and real-world cutting datasets, with generality across cutting velocities and object instances. We then show that Bayesian inference can be performed efficiently by leveraging the differentiability of the simulator, estimating posteriors over hundreds of parameters in a fraction of the time of derivativefree methods. Finally, we illustrate that control parameters in the simulation can be optimized to minimize cutting forces via lateral slicing motions. We publish videos and additional results on our project website at https://diff-cutting-sim.github.io.

show abstract

“…Chebotar et al (2019) randomize simulation parameters and use real world data to update the distribution over simulation parameters while simulatenously learning robotic manipulation tasks. Ramos et al (2019) take a similar approach. Muratore et al (2018) attempt to use real world data to predict transferrability of policies learned in a randomized simulation.…”

Section: Robustness Through Simulator Variancementioning

confidence: 99%

Grounded action transformation for sim-to-real reinforcement learning

et al. 2021

View full text Add to dashboard Cite

Reinforcement learning in simulation is a promising alternative to the prohibitive sample cost of reinforcement learning in the physical world. Unfortunately, policies learned in simulation often perform worse than hand-coded policies when applied on the target, physical system. Grounded simulation learning (gsl) is a general framework that promises to address this issue by altering the simulator to better match the real world (Farchy et al. 2013 in Proceedings of the 12th international conference on autonomous agents and multiagent systems (AAMAS)). This article introduces a new algorithm for gsl—Grounded Action Transformation (GAT)—and applies it to learning control policies for a humanoid robot. We evaluate our algorithm in controlled experiments where we show it to allow policies learned in simulation to transfer to the real world. We then apply our algorithm to learning a fast bipedal walk on a humanoid robot and demonstrate a 43.27% improvement in forward walk velocity compared to a state-of-the art hand-coded walk. This striking empirical success notwithstanding, further empirical analysis shows that gat may struggle when the real world has stochastic state transitions. To address this limitation we generalize gat to the stochasticgat (sgat) algorithm and empirically show that sgat leads to successful real world transfer in situations where gat may fail to find a good policy. Our results contribute to a deeper understanding of grounded simulation learning and demonstrate its effectiveness for applying reinforcement learning to learn robot control policies entirely in simulation.

show abstract

BayesSim: Adaptive Domain Randomization Via Probabilistic Inference for Robotics Simulators

Cited by 89 publications

References 22 publications

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles

DiSECt: A Differentiable Simulation Engine for Autonomous Robotic Cutting

Grounded action transformation for sim-to-real reinforcement learning

Contact Info

Product

Resources

About