On-line learning of temporal state models for flexible objects

Abstract

Abstract-State estimation and control are intimately related processes in robot handling of flexible and articulated objects. While for rigid objects, we can generate a CAD model beforehand and a state estimation boils down to estimation of pose or velocity of the object, in case of flexible and articulated objects, such as a cloth, the representation of the object's state is heavily dependent on the task and execution. For example, when folding a cloth, the representation will mainly depend on the way the folding is executed.In this paper, we address the problem of learning a temporal object model from observations generated during task execution. We use the case of dynamic cloth folding as a proof-ofconcept for our methodology. In cloth folding, the most important information is contained in the temporal structure of the data requiring appropriate representation of the observations, fast state estimation and a suitable prediction mechanism.Our approach is realized through efficient implementation of feature extraction and a generative process model, exploiting recent hardware advances in conjunction with principled probabilistic models. The model is capable of representing the temporal structure of the data and it is robust to noise in the observations. We present results exploiting our model to classify the success of a folding action.

Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey

“…By concatenating the sections, a binary vector describing the three-dimensional garment is formed. Another application for estimating the state of garment can be found in Bergstrom et al (2012), where the focus is on estimating the folding of an article (e.g. a T-shirt and a piece of cloth).…”

Modeling, learning, perception, and control methods for deformable object manipulation

“…Data-driven methods are frequently used, on the basis of visual or multimodal sensing, and are tied to the type of the embodiment used to execute the task. In folding, approaches commonly use some canonical representation and, on the basis of an offline trained model, generate a set of actions that result in some desired folded state (63). (64) addresses rope untangling and models ropes as a chain, learning a function to score the validity of various rope configurations.…”

Enabling garment-agnostic laundry tasks for a Robot Household Companion

“…This work is later improved by Bergstom et al [2] by learning a temporal object model for deformable objects from observations. For this, shapeme histograms are extracted from the image obtained by the robot camera and used as features to both train and test the model.…”