Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience

Abstract

Manipulating deformable objects, such as fabric, is a long standing problem in robotics, with state estimation and control posing a significant challenge for traditional methods. In this paper, we show that it is possible to learn fabric folding skills in only an hour of self-supervised real robot experience, without human supervision or simulation. Our approach relies on fully convolutional networks and the manipulation of visual inputs to exploit learned features, allowing us to create an expressive goal-conditioned pick and place policy that can be trained efficiently with real world robot data only. Folding skills are learned with only a sparse reward function and thus do not require reward function engineering, merely an image of the goal configuration. We demonstrate our method on a set of towel-folding tasks, and show that our approach is able to discover sequential folding strategies, purely from trial-and-error. We achieve state-of-the-art results without the need for demonstrations or simulation, used in prior approaches.

Learning Fabric Manipulation in the Real World with Human Videos

“…However, recent progress in cloth manipulation has been impressive due to advancements in learning-based approaches [12]- [15]. In particular, spatial pick-and-place action spaces allow for the use of Fully Convolutional Networks for learning action affordance heatmaps [1] as well as dynamics models [16] to achieve sample efficiency. Such architectures have also been used to learn fabric smoothing via flinging [17], bimanual stretching [18] and pick-and-place [2], as well as one-step fabric folding policies [19].…”

“…The robot observes the object resting on a flat workspace from an overhead camera, and performs pick-and-place actions, corresponding directly to locations in the image. This is referred to as a spatial action space [25], and is widely used in deformable object manipulation [1], [2], [16], [19]. Our objective is to find the optimal pickand-place action for the task given an image observation.…”

“…Our objective is to find the optimal pickand-place action for the task given an image observation. While prior approaches either perform smoothing alone [4], [26], [27]; assume a pre-folded state for folding [1], [19]; or combine separate smoothing and folding policies with a switching criteria [2], [3]; we aim to learn a single policy that can perform the full smoothing-folding task.…”

“…Prior approaches using only real-world data have utilized random self-supervised robot data collection. However, this can lead to poor performance on long-horizon multi-step tasks [1], or can require large datasets of real-world data and/or additional manual human annotations to achieve good performance [2], [3]. To reduce the amount Robot Executing Learned Task Human Demonstrations I P … Fig.…”

Decentralized Multi-Agent Pursuit Using Deep Reinforcement Learning

“…On the other hand, single-agent reinforcement learning, with the advent of deep-learning based function approximation methods, has enabled learning in challenging domains such as autonomous driving [5], grasping [6] and manipulation [7]. Deep Reinforcement Learning (DRL) has also been successfully applied to multi-agent pursuit-evasion [8]- [11], however most approaches to date did not consider real-world limitations such as local measurements and non-holonomic motion constraints, and did not offer a thorough analysis of the system on operational metrics.…”