Promoting Behavioral Diversity via Multi-Objective/Quality-Diversity Novelty Producing Synaptic Plasticity

Bizzotto, Edoardo; Yaman, Anil; Iacca, Giovanni

doi:10.1109/ssci50451.2021.9659978

Cited by 1 publication

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MCC was demonstrated for the very first time in a maze navigation problem [13]. Mazes are, in fact, a paradigmatic example of tasks with sparse, delayed reward [15], for which various approaches based on quality search [16], [17] or novelty [18] have been proposed. According to the setting proposed in [13], tasks (mazes) are co-evolved with agents (maze navigators) controlled by neural networks.…”

Section: Introductionmentioning

confidence: 99%

Curriculum Learning for Robot Manipulation Tasks With Sparse Reward Through Environment Shifts

Sayar,

Iacca,

Knoll

2024

IEEE Access

Self Cite

View full text Add to dashboard Cite

Multi-goal reinforcement learning (RL) with sparse rewards poses a significant challenge for RL methods. Hindsight experience replay (HER) addresses this challenge by learning from failures and replacing the desired goals with achieved states. However, HER often becomes inefficient when the desired goals are far away from the initial states. This paper introduces co-adapting hindsight experience replay with environment shifts (in short, COHER). COHER generates progressively more complex tasks as soon as the agent's success surpasses a predefined threshold. The generated tasks and agent are coupled to optimize the behavior of the agent within each task-agent pair. We evaluate COHER on various sparse reward robotic tasks that require obstacle avoidance capabilities and compare COHER with hindsight goal generation (HGG), curriculum-guided hindsight experience replay (CHER), and vanilla HER. The results show that COHER consistently outperforms the other methods and that the obtained policies can avoid obstacles without having explicit information about their position. Lastly, we deploy such policies to a real Franka robot for Sim2Real analysis. We observe that the robot can achieve the task by avoiding obstacles, whereas policies obtained with other methods cannot. The videos and code are publicly available at: https://erdiphd.github.io/COHER/.INDEX TERMS Curriculum learning-based reinforcement learning, hindsight experience replay, multi-goal reinforcement learning, robotic control.

show abstract