rl_reach: Reproducible reinforcement learning experiments for robotic reaching tasks

Aumjaud, Pierre; McAuliffe, David; Lera, Francisco Javier Rodríguez; Cardiff, Philip

doi:10.1016/j.simpa.2021.100061

Cited by 6 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Famous problems that did not yet satisfy this requirement have been converted (e.g. Robotics [29], [30]). Moreover, besides theory focused guidelines [11], practical tools like Garage exist to enable reproducible policy training [21].…”

Section: A Why Minimal Traces?mentioning

confidence: 99%

Reliable validation of Reinforcement Learning Benchmarks

Müller-Brockhausen¹,

Plaat²,

Preuß³

2022

Preprint

View full text Add to dashboard Cite

Reinforcement Learning (RL) is one of the most dynamic research areas in Game AI and AI as a whole, and a wide variety of games are used as its prominent test problems. However, it is subject to the replicability crisis that currently affects most algorithmic AI research. Benchmarking in Reinforcement Learning could be improved through verifiable results. There are numerous benchmark environments whose scores are used to compare different algorithms, such as Atari. Nevertheless, reviewers must trust that figures represent truthful values, as it is difficult to reproduce an exact training curve. We propose improving this situation by providing access to the original experimental data to validate study results. To that end, we rely on the concept of minimal traces. These allow re-simulation of action sequences in deterministic RL environments and, in turn, enable reviewers to verify, re-use, and manually inspect experimental results without needing large compute clusters. It also permits validation of presented reward graphs, an inspection of individual episodes, and re-use of result data (baselines) for proper comparison in follow-up papers. We offer plug-and-play code that works with Gym so that our measures fit well in the existing RL and reproducibility eco-system. Our approach is freely available, easy to use, and adds minimal overhead, as minimal traces allow a data compression ratio of up to ≈ 10 4 : 1 (94 GB to 8 MB for Atari Pong) compared to a regular MDP trace used in offline RL datasets. The paper presents proof-ofconcept results for a variety of games.

show abstract

Section: A Why Minimal Traces?mentioning

confidence: 99%

Reliable validation of Reinforcement Learning Benchmarks

Müller-Brockhausen¹,

Plaat²,

Preuß³

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…In the field of robotic manipulation, the reaching task is a well-defined problem that aims to find a valid motion trajectory to put a part of the robot (typically the end effector) to a desired goal position within its workspace. Taking a look at recent publication to the topic of end effector reaching task, a major part of them assigns the orientation of the end effector a subordinate role [18]- [20]. Experiments considering both, position and orientation of end effectors, typically try to solve a kind of grasping, pushing or other related tasks [2], [11], [21].…”

Section: Reaching Taskmentioning

confidence: 99%

Reinforcement Learning for Solving Robotic Reaching Tasks in the Neurorobotics Platform

Szep¹,

Lauenburg²,

Farkas³

et al. 2022

Preprint

View full text Add to dashboard Cite

In recent years, reinforcement learning (RL) has shown great potential for solving tasks in well-defined environments like games or robotics. This paper aims to solve the robotic reaching task in a simulation run on the Neurorobotics Platform (NRP). The target position is initialized randomly and the robot has 6 degrees of freedom. We compare the performance of various state-of-the-art model-free algorithms. At first, the agent is trained on ground truth data from the simulation to reach the target position in only one continuous movement. Later the complexity of the task is increased by using image data as input from the simulation environment. Experimental results show that training efficiency and results can be improved with appropriate dynamic training schedule function for curriculum learning.

show abstract

“…A reacher task [28] that uses an ABB IRB 120 was implemented to test the library, this task is explained in a detailed way in Section IV. The entire process needed to create the RobotEnv, TaskEnv, the RL parameter YAML file and the script needed to train the policy is available in the library documentation 2 .…”

Section: E Example Environmentmentioning

confidence: 99%

“…The reacher task [28] is a typical application of RL algorithms with robotic manipulators, where the manipulator must achieve a spatial position or pose. The reacher task can be viewed as a learning problem where the RL is going to learn the robot Jacobian matrix and achieve a spatial position through multiple iterations.…”

Section: A Manipulator Reachermentioning

confidence: 99%

FRobs_RL: A Flexible Robotics Reinforcement Learning Library

Fajardo

Roldan

Realpe

et al. 2022

2022 IEEE 18th International Conference on Automation Science and Engineering (CASE)

View full text Add to dashboard Cite

Reinforcement learning (RL) has become an interesting topic in robotics applications as it can solve complex problems in specific scenarios. The small amount of RL-tools focused on robotics, plus the lack of features such as easy transfer of simulated environments to real hardware, are obstacles to the widespread use of RL in robotic applications. FRobs RL is a Python library that aims to facilitate the implementation, testing, and deployment of RL algorithms in intelligent robotic applications using robot operating system (ROS), Gazebo, and OpenAI Gym. FRobs RL provides an Application Programming Interface (API) to simplify the creation of RL environments, where users can import a wide variety of robot models as well as different simulated environments. With the FRobs RL library, users do not need to be experts in ROS, Gym, or Gazebo to create a realistic RL application. Using the library, we created and tested two environments containing common robotic tasks; one is a reacher task using a robotic manipulator, and the other is a mapless navigation task using a mobile robot. The library is available in GitHub 1 .

show abstract

rl_reach: Reproducible reinforcement learning experiments for robotic reaching tasks

Cited by 6 publications

References 10 publications

Reliable validation of Reinforcement Learning Benchmarks

Reliable validation of Reinforcement Learning Benchmarks

Reinforcement Learning for Solving Robotic Reaching Tasks in the Neurorobotics Platform

FRobs_RL: A Flexible Robotics Reinforcement Learning Library

Contact Info

Product

Resources

About