Efficient Self-Supervised Data Collection for Offline Robot Learning

Endrawis, Shadi; Leibovich, Gal; Jacob, Guy; Novik, Gal; Tamar, Aviv

doi:10.1109/icra48506.2021.9561097

Cited by 9 publications

(6 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-task Offline RL. Recent works proposed to learn multiple tasks from pre-collected datasets, starting with methods [32] that generate goals to improve the offline data collection process in a selfsupervised way. This connection has also been studied in the supervised setting [9,33] and when learning hierarchical policies [10].…”

Section: Related Workmentioning

confidence: 99%

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping

Mezghani¹,

Sukhbaatar²,

Bojanowski³

et al. 2023

Preprint

View full text Add to dashboard Cite

Developing agents that can execute multiple skills by learning from pre-collected datasets is an important problem in robotics, where online interaction with the environment is extremely time-consuming. Moreover, manually designing reward functions for every single desired skill is prohibitive. Prior works [1, 2] targeted these challenges by learning goal-conditioned policies from offline datasets without manually specified rewards, through hindsight relabeling. These methods suffer from the issue of sparsity of rewards, and fail at long-horizon tasks. In this work, we propose a novel self-supervised learning phase on the pre-collected dataset to understand the structure and the dynamics of the model, and shape a dense reward function for learning policies offline. We evaluate our method on three continuous control tasks, and show that our model significantly outperforms existing approaches [1, 2], especially on tasks that involve long-term planning.

show abstract

Section: Related Workmentioning

confidence: 99%

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping

Mezghani¹,

Sukhbaatar²,

Bojanowski³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…A straightforward implementation of Data Generation consists in collecting data with the random uniform policy. However, this approach is not guaranteed to explore the state space effectively (Mutti et al, 2021;Endrawis et al, 2021). In the pipeline, we consider the state-of-the-art solutions proposed by Pathak et al (2019), and Mutti et al (2021).…”

Section: Data Generationmentioning

confidence: 99%

ARLO: A Framework for Automated Reinforcement Learning

Mussi¹,

Lombarda²,

Metelli³

et al. 2022

Preprint

View full text Add to dashboard Cite

Automated Reinforcement Learning (AutoRL) is a relatively new area of research that is gaining increasing attention. The objective of AutoRL consists in easing the employment of Reinforcement Learning (RL) techniques for the broader public by alleviating some of its main challenges, including data collection, algorithm selection, and hyper-parameter tuning. In this work, we propose a general and flexible framework, namely ARLO: Automated Reinforcement Learning Optimizer, to construct automated pipelines for AutoRL. Based on this, we propose a pipeline for offline and one for online RL, discussing the components, interaction, and highlighting the difference between the two settings. Furthermore, we provide a Python implementation of such pipelines, released as an open-source library. Our implementation has been tested on an illustrative LQG domain and on classic MuJoCo environments, showing the ability to reach competitive performances requiring limited human intervention. We also showcase the full pipeline on a realistic dam environment, automatically performing the feature selection and the model generation tasks.

show abstract

“…In [23], high-quality and diverse data are obtained with human demonstrations, which proves to broadly improve the performance of the robotic learning tasks. To reduce manual effort, efficient self-supervised data collection has been introduced for offline robot learning with task-relevant loss functions [24], [25]. Alternatively, online data collection without manual intervention can be achieved through robot interactions with the environments [26].…”

Section: B Data Collection and Sim-to-realmentioning

confidence: 99%

Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes

Xie¹,

Yu²,

Xu³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper proposes a learning-based visual pegin-hole that enables training with several shapes in simulation, and adapting to arbitrary unseen shapes in real world with minimal sim-to-real cost. The core idea is to decouple the generalization of the sensory-motor policy to the design of a fast-adaptable perception module and a simulated generic policy module. The framework consists of a segmentation network (SN), a virtual sensor network (VSN), and a controller network (CN). Concretely, the VSN is trained to measure the pose of the unseen shape from a segmented image. After that, given the shapeagnostic pose measurement, the CN is trained to achieve generic peg-in-hole. Finally, when applying to real unseen holes, we only have to fine-tune the SN required by the simulated VSN+CN. To further minimize the transfer cost, we propose to automatically collect and annotate the data for the SN after one-minute human teaching. Simulated and real-world results are presented under the configurations of eye-to/in-hand. An electric vehicle charging system with the proposed policy inside achieves a 10/10 success rate in 2∼3s, using only hundreds of auto-labeled samples for the SN transfer.

show abstract

Efficient Self-Supervised Data Collection for Offline Robot Learning

Cited by 9 publications

References 16 publications

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping

Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping

ARLO: A Framework for Automated Reinforcement Learning

Learning A Simulation-based Visual Policy for Real-world Peg In Unseen Holes

Contact Info

Product

Resources

About