GOATS: Goal Sampling Adaptation for Scooping with Curriculum Reinforcement Learning

Abstract

In this work, we first formulate the problem of goal-conditioned robotic water scooping with reinforcement learning. This task is challenging due to the complex dynamics of fluid and multi-modal goal-reaching. The policy is required to achieve both position goals and water amount goals, which leads to a large convoluted goal state space. To address these challenges, we introduce Goal Sampling Adaptation for Scooping (GOATS), a curriculum reinforcement learning method that can learn an effective and generalizable policy for robot scooping tasks. Specifically, we use a goal-factorized reward formulation and interpolate position goal distributions and amount goal distributions to create curriculum through the learning process. As a result, our proposed method can outperform the baselines in simulation and achieves 5.46% and 8.71% amount errors on bowl scooping and bucket scooping tasks, respectively, under 1000 variations of initial water states in the tank and a large goal state space. Besides being effective in simulation environments, our method can efficiently generalize to noisy real-robot water-scooping scenarios with different physical configurations and unseen settings, demonstrating superior efficacy and generalizability. The videos of this work are available on our project page: https://sites.google.com/view/goatscooping.• To the best of our knowledge, we are the first to formu-

Ricci Planner: Zero-Shot Transfer for Goal-Conditioned Reinforcement Learning via Geometric Flow

“…Consequently, there is a need for novel formulations and methods that account for such characteristics of environments. To address this challenge, a new formulation has been proposed, called the goal augmented Markov decision process (GA-MDP) [1], and active research has been conducted under the name of goal conditioned reinforcement learning (GCRL) [1], [15], [32] in the literature.…”

TacGNN: Learning Tactile-Based In-Hand Manipulation With a Blind Robot Using Hierarchical Graph Neural Network

“…As the mechanism design of the hand becomes increasingly dedicated and complex, it is getting harder to design an analytical model [14] to control the hands for various tasks. The rise of deep learning provides researchers with huge potential to achieve dexterous manipulation by experiences like humans [5], [15], [16]. Existing work has been using the benefit of visual data from a camera or motion capture system for dexterous manipulation [4], [11], [14].…”

Learning Shape Control of Elastoplastic Deformable Linear Objects

“…Since the first results in this paper were presented [22], there has been some work on GCRL for non-rigid material manipulation that is important to highlight here. Niu et al [23] presented GOATS -Goal Sampling Adaptation for Scooping, an algorithm for goal-sampling based on the curriculum learning idea, i.e. starting with simpler goals and progressively increasing to more complex ones.…”