dm_control: Software and tasks for continuous control

Tassa, Yuval; Tunyasuvunakool, Saran; Muldal, Alistair; Doron, Yotam; Trochim, Piotr; Liu, Siqi; Bohez, Steven; Merel, Josh; Erez, Tom; Lillicrap, Timothy P.; Heess, Nicolas

doi:10.1016/j.simpa.2020.100022

Cited by 104 publications

(52 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The simulated environments include Meta-World (Yu et al, 2020) introduced to benchmark meta-reinforcement learning and multi-task learning, Sokoban (Racanière et al, 2017) proposed as a planning problem, BabyAI (Chevalier-Boisvert et al, 2018) for language instruction following in grid-worlds, the DM Control Suite (Tunyasuvunakool et al, 2020) for continuous control, as well as DM Lab (Beattie et al, 2016) designed to teach agents navigation and 3D vision from raw pixels with an egocentric viewpoint. We also use the Arcade Learning Environment (Bellemare et al, 2013) with classic Atari games (we use two sets of games that we call ALE Atari and ALE Atari Extended, see Section F.1 for details).…”

Section: Simulated Control Tasksmentioning

confidence: 99%

See 1 more Smart Citation

A Generalist Agent

Reed¹,

Żołna²,

Parisotto³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.

show abstract

Section: Simulated Control Tasksmentioning

confidence: 99%

“…The DeepMind Control Suite Tunyasuvunakool et al, 2020) is a set of physicsbased simulation environments. For each task in the control suite we collect two disjoint sets of data, one using only state features and another using only pixels.…”

Section: F Data Collection Detailsmentioning

confidence: 99%

A Generalist Agent

Reed¹,

Żołna²,

Parisotto³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…In contrast, RL methods are routinely used for a variety of other problems, such as robotics (Tunyasuvunakool et al, 2020) autonomous driving (Sallab et al, 2017), and smart building energy management (Yu et al, 2021). The main attraction of BSD is the principled nature of the solution.…”

Section: Discussionmentioning

confidence: 99%

“…Environments can be based on simulations. For example, popular RL applications with simulation-based environments include Atari video-games (Mnih et al, 2013), robotic tasks (Tunyasuvunakool et al, 2020) and autonomous driving (Sallab et al, 2017;Wurman et al, 2022).…”

Section: Introductionmentioning

confidence: 99%

A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning

Tec,

Duan,

Müller

2022

Preprint

View full text Add to dashboard Cite

Reinforcement Learning (RL) is a computational approach to reward-driven learning in sequential decision problems. It implements the discovery of optimal actions by learning from an agent interacting with an environment rather than from supervised data. We contrast and compare RL with traditional sequential design, focusing on simulation-based Bayesian sequential design (BSD). Recently, there has been an increasing interest in RL techniques for healthcare applications. We introduce two related applications as motivating examples. In both applications, the sequential nature of the decisions is restricted to sequential stopping. Rather than a comprehensive survey, the focus of the discussion is on solutions using standard tools for these two relatively simple sequential stopping problems. Both problems are inspired by adaptive clinical trial design. We use examples to explain the terminology and mathematical background that underlie each framework and map one to the other. The implementations and results illustrate the many similarities between RL and BSD. The results motivate the discussion of the potential strengths and limitations of each approach.

show abstract

“…Thanks to the recent progress of physics simulation (Todorov et al 2012;Coumans and Bai 2016;Erez et al 2015), it has drawn increasing interest to build full-physics robot simulation environment (Urakami et al 2019;Tunyasuvunakool et al 2020;Zhu et al 2020;James et al 2020;Mu et al 2021). Compared to robot simulation with abstract action (Kolve et al 2017;Savva et al 2019;, full-physic robot simulation supports low-level policy learning that could be transferred to real world.…”

Section: Robot Simulationmentioning

confidence: 99%

Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation

Zhang¹,

Chen²,

Xiang³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper, we focus on the simulation of active stereovision depth sensors, which are popular in both academic and industry communities. Inspired by the underlying mechanism of the sensors, we designed a fully physics-grounded simulation pipeline, which includes material acquisition, ray tracing based infrared (IR) image rendering, IR noise simulation, and depth estimation. The pipeline is able to generate depth maps with material-dependent error patterns similar to a real depth sensor. We conduct extensive experiments to show that perception algorithms and reinforcement learning policies trained in our simulation platform could transfer well to real world test cases without any fine-tuning. Furthermore, due to the high degree of realism of this simulation, our depth sensor simulator can be used as a convenient testbed to evaluate the algorithm performance in the real world, which will largely reduce the human effort in developing robotic algorithms. The entire pipeline has been integrated into the SAPIEN simulator and is open-sourced to promote the research of vision and robotics communities.

show abstract

dm_control: Software and tasks for continuous control

Cited by 104 publications

References 11 publications

A Generalist Agent

A Generalist Agent

A Comparative Tutorial of Bayesian Sequential Design and Reinforcement Learning

Close the Optical Sensing Domain Gap by Physics-Grounded Active Stereo Sensor Simulation

Contact Info

Product

Resources

About