Feedback-based Fabric Strip Folding

Abstract

Accurate manipulation of a deformable body such as a piece of fabric is difficult because of its many degrees of freedom and unobservable properties affecting its dynamics. To alleviate these challenges, we propose the application of feedback-based control to robotic fabric strip folding. The feedback is computed from the low dimensional state extracted from a camera image. We trained the controller using reinforcement learning in simulation which was calibrated to cover the real fabric strip behaviors. The proposed feedback-based folding was experimentally compared to two state-of-the-art folding methods and our method outperformed both of them in terms of accuracy.

Modeling, learning, perception, and control methods for deformable object manipulation

“…Because this simulator originally targeted modeling of articulated robots, it provides useful interfaces for expediting robot modeling and learning algorithms. Its deformable dynamics modeling has been applied in learning a cloth folding primitive (33) and rope manipulation planning (6).…”

Disruption-Resistant Deformable Object Manipulation on Basis of Online Shape Estimation and Prediction-Driven Trajectory Correction

“…Limitations of the above approaches are lack of on-line trajectory correction during manipulation, and the assumption of fixed goal states. Petrík and Kyrki [15] realise fine feedback control with robustness to material variation for the constrained case of folding a strip of cloth in two, using a reinforcement learning approach with a low-dimensional state representation. Hu et al [16] tightly interlink recognition and motion generation using machine learning techniques operating on raw sensor data.…”

Enabling garment-agnostic laundry tasks for a Robot Household Companion

“…In recent work, Petrik et al focus on understanding the physical properties and state of a cloth during folding. In [67], they focus on accurate folding of a single cloth strip by taking into account the dynamics and behavior of the cloth though a vision feedback-based controller. The controller is trained using reinforcement learning in a calibrated simulation matching the real cloth strip properties.…”

“…To dynamically control the cloth during the folding operation, so that it can reach the correct contact location and proceed with the rest of the folding operation, a controller is proposed based on prior work by Vladimír Petrík et al [67]. The proposed controller is currently being developed in collaboration with the original authors, a collaboration materialized through a research stay at the Czech Technical University (CTU) of Prague.…”

“…The work proposed in this thesis, developed in collaboration with CTU Prague, improves their original garment state estimation [67] by using a Deep Neural Network to obtain a state vector that describes the cloth strip state with a higher detail (723 dimensions vs the original 3 dimensions). The main objective of this new state estimation method is to improve the accuracy of the neural controller policy by increasing the amount of information available about the garment state, extracted visually from RGB images during the cloth strip manipulation.…”