Making Curiosity Explicit in Vision-based RL

Aljalbout, Elie; Ulmer, Maximilian; Triebel, Rudolph

doi:10.48550/arxiv.2109.13588

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach takes advantage of the offpolicy property of most state-of-the-art RL algorithms and trains a separate curious policy based on the SRL error. A preliminary version of this work can be found in [3]. Our experiments show that the proposed method encourages the visitation of SRL-problematic states.…”

Section: Introductionmentioning

confidence: 98%

Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning

Aljalbout¹,

Ulmer²,

Triebel³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Vision-based reinforcement learning (RL) is a promising approach to solve control tasks involving images as the main observation. State-of-the-art RL algorithms still struggle in terms of sample efficiency, especially when using image observations. This has led to increased attention on integrating state representation learning (SRL) techniques into the RL pipeline. Work in this field demonstrates a substantial improvement in sample efficiency among other benefits. However, to take full advantage of this paradigm, the quality of samples used for training plays a crucial role. More importantly, the diversity of these samples could affect the sample efficiency of vision-based RL, but also its generalization capability. In this work, we present an approach to improve sample diversity for state representation learning. Our method enhances the exploration capability of RL algorithms, by taking advantage of the SRL setup. Our experiments show that our proposed approach boosts the visitation of problematic states, improves the learned state representation, and outperforms the baselines for all tested environments. These results are most apparent for environments where the baseline methods struggle. Even in simple environments, our method stabilizes the training, reduces the reward variance, and promotes sample efficiency.

show abstract

Section: Introductionmentioning

confidence: 98%

Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning

Aljalbout¹,

Ulmer²,

Triebel³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Sample inefficiency is most commonly attributed to the complexity and noise encountered in sensory information processing. Solutions to the problem range from including pretrained perception modules [12] in the learning pipeline to integrating self-supervised state representation learning objectives into task learning [13,10,4,14]. Safe exploration and environment resetting are rarely mentioned in publications and temporary solutions include engineering the environment or having a human manually stop the robot in dangerous situations and reset the environment at the end of each trial.…”

Section: Introductionmentioning

confidence: 99%

Dual-Arm Adversarial Robot Learning

Aljalbout

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Robot learning is a very promising topic for the future of automation and machine intelligence. Future robots should be able to autonomously acquire skills, learn to represent their environment, and interact with it. While these topics have been explored in simulation, real-world robot learning research seems to be still limited. This is due to the additional challenges encountered in the real-world, such as noisy sensors and actuators, safe exploration, non-stationary dynamics, autonomous environment resetting as well as the cost of running experiments for long periods of time. Unless we develop scalable solutions to these problems, learning complex tasks involving hand-eye coordination and rich contacts will remain an untouched vision that is only feasible in controlled lab environments. We propose dual-arm settings as platforms for robot learning. Such settings enable safe data collection for acquiring manipulation skills as well as training perception modules in a robot-supervised manner. They also ease the processes of resetting the environment. Furthermore, adversarial learning could potentially boost the generalization capability of robot learning methods by maximizing the exploration based on game-theoretic objectives while ensuring safety based on collaborative task spaces. In this paper, we will discuss the potential benefits of this setup as well as the challenges and research directions that can be pursued.

show abstract

Making Curiosity Explicit in Vision-based RL

Cited by 2 publications

References 0 publications

Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning

Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning

Dual-Arm Adversarial Robot Learning

Contact Info

Product

Resources

About