Automatic 3-D Manipulation of Soft Objects by Robotic Arms With an Adaptive Deformation Model

IEEE Robot. Autom. Lett.

2018

In this paper, we present a general approach to automatically visual-servo control the position and shape of a deformable object whose deformation parameters are unknown. The servo-control is achieved by online learning a model mapping between the robotic end-effector's movement and the object's deformation measurement. The model is learned using the Gaussian Process Regression (GPR) to deal with its highly nonlinear property, and once learned, the model is used for predicting the required control at each time step. To overcome GPR's high computational cost while dealing with long manipulation sequences, we implement a fast online GPR by selectively removing uninformative observation data from the regression process. We validate the performance of our controller on a set of deformable object manipulation tasks and demonstrate that our method can achieve effective and accurate servo-control for general deformable objects with a wide variety of goal settings. Experiment videos are available at https://sites.google.com/view/mso-fogpr.

Section: Introductionmentioning

confidence: 90%

Three-Dimensional Deformable Object Manipulation Using Fast Online Gaussian Process Regression

Sun

IEEE Robot. Autom. Lett.

2018

“…Recently, [31,32,33] proposed using non-rigid registration to transfer human demonstrations of cloth manipulations to real robots and [34] required an adaptive cloth simulator to predict the future state of a cloth. However, these methods require the knowledge of full 3D cloth geometries, which are not available in our applications.…”

Section: Related Workmentioning

confidence: 99%

Cloth Manipulation Using Random-Forest-Based Imitation Learning

Jia

IEEE Robot. Autom. Lett.

et al. 2019

We present a novel approach for manipulating high-DOF deformable objects such as cloth. Our approach uses a random-forest-based controller that maps the observed visual features of the cloth to an optimal control action of the manipulator. The topological structure of this random-forest is determined automatically based on the training data, which consists of visual features and control signals. The training data is constructed online using an imitation learning algorithm. We have evaluated our approach on different cloth manipulation benchmarks such as flattening, folding, and twisting. In all these tasks, we have observed convergent behavior for the random-forest. On convergence, the random-forest-based controller exhibits superior robustness to observation noise compared with other techniques such as convolutional neural networks and nearest neighbor searches.

“…Robotic manipulation of general deformable objects relies on a combination of different sensor measurements. The RGB images or RGB-Depth data are widely used for deformable object manipulation [1], [4], [9]. Fiducial markers can also be printed on the deformable object to improve the manipulation performance [16].…”

Section: B Deformable Object Manipulationmentioning

confidence: 99%

Manipulating Highly Deformable Materials Using a Visual Feedback Dictionary

Jia

2018 IEEE International Conference on Robotics and Automation (ICRA)

et al. 2018

The complex physical properties of highly deformable materials such as clothes pose significant challenges fanipulation systems. We present a novel visual feedback dictionary-based method for manipulating defoor autonomous robotic mrmable objects towards a desired configuration. Our approach is based on visual servoing and we use an efficient technique to extract key features from the RGB sensor stream in the form of a histogram of deformable model features. These histogram features serve as high-level representations of the state of the deformable material. Next, we collect manipulation data and use a visual feedback dictionary that maps the velocity in the high-dimensional feature space to the velocity of the robotic end-effectors for manipulation. We have evaluated our approach on a set of complex manipulation tasks and humanrobot manipulation tasks on different cloth pieces with varying material characteristics.