Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments

Mittal, Mayank; Yu, Calvin C.K.; Yu, Qinxi; Liu, Jingzhou; Rudin, Nikita; Hoeller, David; Yuan, Jia Lin; Singh, Ritvik; Guo, Yunrong; Mazhar, Hammad; Mandlekar, Ajay; Babich, B. N.; State, Gavriel; Hutter, Marco; Garg, Animesh

doi:10.1109/lra.2023.3270034

Cited by 36 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some examples of physics-based simulators include Gazebo, 75 MuJoCo, 76 and NVIDIA Isaac Sim. 77,78 Recently, NVIDIA Isaac Lab, 77 a modular framework on top of Isaac Sim, was introduced to simplify common workflows for robot learning that is pivotal for robot foundation model training. Some examples of robotic simulation environments and benchmarks are iGibson, 79 MetaWorld, 80 and BEHAVIOR-1K.…”

Section: Manipulation Skill Learning and Digital Twinmentioning

confidence: 99%

Self-Driving Laboratories for Chemistry and Materials Science

Tom,

Schmid,

Baird

et al. 2024

Preprint

View full text Add to dashboard Cite

Self-driving laboratories (SDLs) promise an accelerated application of the scientific method. Through the automation of experimental workflows, along with the autonomization of experiment planning, SDLs hold the potential to greatly accelerate research in chemistry and materials discovery. This review article provides an in-depth analysis of the state-of-the-art in SDL technology, its applications across various scientific disciplines, and the potential implications for research, and industry. This review additionally provides an overview of the enabling technologies for SDLs, including their hardware, software, and integration with laboratory infrastructure. Most importantly, this review explores the diverse range of scientific domains where SDLs have made significant contributions, from drug discovery and materials science to genomics and chemistry. We provide a comprehensive review of existing real-world examples of SDLs, their different levels of automation, and the challenges and limitations associated with each domain.

show abstract

Section: Manipulation Skill Learning and Digital Twinmentioning

confidence: 99%

Self-Driving Laboratories for Chemistry and Materials Science

Tom,

Schmid,

Baird

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Based on our empirical tests, training in Isaac Sim can reduce training time by approximately one to two orders of magnitude, depending on the specific task. To reduce development costs, we utilize the recently introduced Isaac Orbit [ 28 ], a modular environment for robot learning based on Isaac Sim. This modular environment defines common data structures and operational interfaces for quadruped robots, allowing for easy deployment of quadruped robots in the simulation environment with minor modifications.…”

Section: Experiments and Analysismentioning

confidence: 99%

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks

Wan,

Luo,

Meng

et al. 2023

Sensors

View full text Add to dashboard Cite

In the field of quadruped robots, the most classic motion control algorithm is based on model prediction control (MPC). However, this method poses challenges as it necessitates the precise construction of the robot’s dynamics model, making it difficult to achieve agile movements similar to those of a biological dog. Due to these limitations, researchers are increasingly turning to model-free learning methods, which significantly reduce the difficulty of modeling and engineering debugging and simultaneously reduce real-time optimization computational burden. Inspired by the growth process of humans and animals, from learning to walk to fluent movements, this article proposes a hierarchical reinforcement learning framework for the motion controller to learn some higher-level tasks. First, some basic motion skills can be learned from motion data captured from a biological dog. Then, with these learned basic motion skills as a foundation, the quadruped robot can focus on learning higher-level tasks without starting from low-level kinematics, which saves redundant training time. By utilizing domain randomization techniques during the training process, the trained policy function can be directly transferred to a physical robot without modification, and the resulting controller can perform more biomimetic movements. By implementing the method proposed in this article, the agility and adaptability of the quadruped robot can be maximally utilized to achieve efficient operations in complex terrains.

show abstract

SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding

Jia,

Chen,

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments

Cited by 36 publications

References 19 publications

Self-Driving Laboratories for Chemistry and Materials Science

Self-Driving Laboratories for Chemistry and Materials Science

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks

SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding

Contact Info

Product

Resources

About