Learning low level skills from scratch for humanoid robot soccer using deep reinforcement learning

Abreu, Miguel; Lau, Nuno; Sousa, Armando; Reis, Luís Paulo

doi:10.1109/icarsc.2019.8733632

Cited by 25 publications

(15 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our learning framework employs the Proximal Policy Optimization (PPO) algorithm, introduced by Schulman et al [37], which was chosen due to its success in optimizing low-level skills concerning the NAO robot [21,38,39,40,41], and highlevel skills [42], where it outperformed other algorithms such as TRPO or DDPG. The chosen implementation uses the clipped surrogate objective:…”

Section: Learning Frameworkmentioning

confidence: 99%

“…• Performance: to have an entirely fair comparison, the performance of our framework should be compared with other frameworks in the same scenario and simulator. To do so, we took into consideration the maximum forward speed, and our proposed framework provides a faster walk than the agents in [45,50,46,25,19] and slower than [51,21,38,41]. However, some of the faster examples are solely focused on sprinting forward, without the basic ability of changing direction [21,38,41].…”

Section: Featuresmentioning

confidence: 99%

“…To do so, we took into consideration the maximum forward speed, and our proposed framework provides a faster walk than the agents in [45,50,46,25,19] and slower than [51,21,38,41]. However, some of the faster examples are solely focused on sprinting forward, without the basic ability of changing direction [21,38,41]. The comparison results are summarized in Table 5.…”

Section: Featuresmentioning

confidence: 99%

See 2 more Smart Citations

Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach

Kasaei,

Abreu,

Lau

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper proposes a modular framework to generate robust biped locomotion using a tight coupling between an analytical walking approach and deep reinforcement learning. This framework is composed of six main modules which are hierarchically connected to reduce the overall complexity and increase its flexibility. The core of this framework is a specific dynamics model which abstracts a humanoid's dynamics model into two masses for modeling upper and lower body. This dynamics model is used to design an adaptive reference trajectories planner and an optimal controller which are fully parametric. Furthermore, a learning framework is developed based on Genetic Algorithm (GA) and Proximal Policy Optimization (PPO) to find the optimum parameters and to learn how to improve the stability of the robot by moving the arms and changing its center of mass (COM) height. A set of simulations are performed to validate the performance of the framework using the official RoboCup 3D League simulation environment. The results validate the performance of the framework, not only in creating a fast and stable gait but also in learning to improve the upper body efficiency.

show abstract

Section: Learning Frameworkmentioning

confidence: 99%

Section: Featuresmentioning

confidence: 99%

Section: Featuresmentioning

confidence: 99%

See 1 more Smart Citation

Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach

Kasaei,

Abreu,

Lau

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…First, in [17], the action space comprises two controllable, abstract, and consequently discrete commands: dash, to get closer to the ball, and kick, to push the ball. Second, in [18], the objective is to produce continuous actions, which are considered as low-level.…”

Section: Related Workmentioning

confidence: 99%

“…It is also important to distinguish the concept of learning at a high or at a low level of abstraction. For instance, the authors of [17] and [18] separate this in two different applications in a simulated environment. First, in [17], the action space comprises two controllable, abstract, and consequently discrete commands: dash, to get closer to the ball, and kick, to push the ball.…”

Section: Related Workmentioning

confidence: 99%

A Framework for Studying Reinforcement Learning and Sim-to-Real in Robot Soccer

Bassani,

Delgado,

Junior

et al. 2020

Preprint

View full text Add to dashboard Cite

This article introduces an open framework, called VSSS-RL, for studying Reinforcement Learning (RL) and sim-to-real in robot soccer, focusing on the IEEE Very Small Size Soccer (VSSS) league. We propose a simulated environment in which continuous or discrete control policies can be trained to control the complete behavior of soccer agents and a sim-to-real method based on domain adaptation to adapt the obtained policies to real robots. Our results show that the trained policies learned a broad repertoire of behaviors that are difficult to implement with handcrafted control policies. With VSSS-RL, we were able to beat human-designed policies in the 2019 Latin American Robotics Competition (LARC), achieving 4th place out of 21 teams, being the first to apply Reinforcement Learning (RL) successfully in this competition. Both environment and hardware specifications are available open-source to allow reproducibility of our results and further studies.

show abstract

Learning to Run Faster in a Humanoid Robot Soccer Environment Through Reinforcement Learning

Abreu

Reis

Lau

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Learning low level skills from scratch for humanoid robot soccer using deep reinforcement learning

Cited by 25 publications

References 5 publications

Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach

Robust Biped Locomotion Using Deep Reinforcement Learning on Top of an Analytical Control Approach

A Framework for Studying Reinforcement Learning and Sim-to-Real in Robot Soccer

Learning to Run Faster in a Humanoid Robot Soccer Environment Through Reinforcement Learning

Contact Info

Product

Resources

About