B-Human 2019 – Complex Team Play Under Natural Lighting Conditions

Röfer, Thomas; Laue, Tim; Felsch, Gerrit; Hasselbring, Arne; Haß, Tim; Oppermann, Jan Patrick; Reichenberg, Philip; Schrader, Nicole

doi:10.1007/978-3-030-35699-6_52

Cited by 8 publications

(5 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In 10 trials, the agent first walked to the path between the ball and its own goal, to block a potential shot from the opponent, before turning toward the ball and approaching the ball and opponent. This type of defensive behavior is manually implemented in some RoboCup teams to defend the goal against an attacker with possession (36). Our learned policy, in contrast, discovered this tactic "on its own" by optimizing for task reward (which includes minimizing opponent scoring) rather than via manual specification.…”

Section: Opponent Awarenessmentioning

confidence: 99%

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Haarnoja,

Moran,

Lever

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent’s tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

show abstract

Section: Opponent Awarenessmentioning

confidence: 99%

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Haarnoja,

Moran,

Lever

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

show abstract

“…Reinforcement learning is an area that has received attention since the early days of RoboCup to tackle some of the main technical challenges, which emerge in various of its football leagues, in isolation. These works typically focus on the handling of large state-action spaces (134,135), skill learning (136)(137)(138)(139)(140)(141), the keep-away and half-field offense tasks and multi-agent coordination (142)(143)(144)(145)(146)(147)(148)(149)(150), grounded simulation learning for improved skills (151-154) (e.g. in sim-toreal and back), skill learning in 3D humanoid football (155,156), and deep reinforcement learning for parameterised action spaces (157).…”

Section: Robocup and Simulated Footballmentioning

confidence: 99%

“…Yet, despite these examples of applications of reinforcement learning in the RoboCup domain, many successful recent RoboCup competition entries learn or optimize only a subset of the components of the control architecture (e.g. 141,158). One successful RoboCup approach related to our method is Layered Learning (29,159) which uses reinforcement learning at multiple levels of a pre-defined hierarchy of skills, from individual ball interaction to multi-agent behaviours such as pass selection.…”

Section: Robocup and Simulated Footballmentioning

confidence: 99%

From Motor Control to Team Play in Simulated Humanoid Football

Liu¹,

Lever²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Intelligent behaviour in the physical world exhibits structure at multiple spatial and temporal scales. Although movements are ultimately executed at the level of instantaneous muscle tensions or joint torques, they must be selected so as to serve goals defined on much longer timescales, and in terms of relations that extend far beyond the body itself, ultimately involving coordination with other agents. Recent research in artificial intelligence has shown the promise of learning-based approaches to the respective problems of complex movement, longer-term planning, and multi-agent coordination. However, there is limited research aimed at their integration. We study this problem by training teams of physically simulated humanoid avatars to play football in a realistic virtual environment. We develop a method that combines imitation learning, single-and multi-agent reinforcement learning and population-based training, and makes use of transferable representations of behaviour for decision making at different levels of abstraction. In a sequence of training stages, players first learn to control a fully articulated body to perform realistic, human-like movements such as running and turning; they then acquire mid-level football skills such as dribbling and shooting; finally, they develop awareness of others and learn to play as a team, successfully bridging the gap between low-level motor control at a time scale of milliseconds, and coordinated goal-directed behaviour as a team at the timescale of tens of seconds. We investigate the emergence of behaviours at different levels of abstraction, as well as the representations that underlie these behaviours using several analysis techniques, including statistics from real-world sports analytics. Our work constitutes a complete demonstration of integrated decision-making at multiple scales in a physically embodied multi-agent setting. We provide footage of the learned football skills in the supplementary video. 1

show abstract

“…Implementations for efficient neural network inference on the NAO platform have been provided by [11] as well as by [12]. The classification of preprocessed image patches, e. g. for ball classification, has become the de facto standard, one example out of many is given in [13]. A Deep Learning approach for processing the whole image for the purpose of robot detection has been presented by [14].…”

Section: Scientific and Technical Challengesmentioning

confidence: 99%

Let Your Robot Go – Challenges of a Decentralized Remote Robot Competition

Laue

Moos

Göttsch

2021

2021 European Conference on Mobile Robots (ECMR)

Self Cite

View full text Add to dashboard Cite

Holding an international robot competition, such as the RoboCup, usually requires many people and their robots to come together in one place for several days. In case of international travel restrictions, such an event cannot take place in its intended way. In this paper, we describe the German Open Replacement Event (GORE), a robot tournament for RoboCup Standard Platform League teams, which was organized in a decentralized way and required remote participation. Furthermore, we provide insights into the specific technical and scientific challenges the participating teams had to face, along with an outlook on implications for future robot competitions.

show abstract

B-Human 2019 – Complex Team Play Under Natural Lighting Conditions

Cited by 8 publications

References 4 publications

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

From Motor Control to Team Play in Simulated Humanoid Football

Let Your Robot Go – Challenges of a Decentralized Remote Robot Competition

Contact Info

Product

Resources

About