A 3D descriptor to detect task-oriented grasping points in clothing

Abstract

Manipulating textile objects with a robot is a challenging task, especially because the garment perception is difficult due to the endless configurations it can adopt, coupled with a large variety of colors and designs. Most current approaches follow a multiple re-grasp strategy, in which clothes are sequentially grasped from different points until one of them yields a recognizable configuration. In this work we propose a method that combines 3D and appearance information to directly select a suitable grasping point for the task at hand, which in our case consists of hanging a shirt or a polo shirt from a hook. Our method follows a coarse-to-fine approach in which, first, the collar of the garment is detected and, next, a grasping point on the lapel is chosen using a novel 3D descriptor.In contrast to current 3D descriptors, ours can run in real time, even when it needs to be densely computed over the input image. Our central idea is to take advantage of the structured nature of range images that most depth sensors provide and, by exploiting integral imaging, achieve speed-ups of two orders of magnitude with respect to competing approaches, while maintaining performance. This makes it especially adequate for robotic applications as we thoroughly demonstrate in the experimental section.

Hand-Object Interaction: From Human Demonstrations to Robot Manipulation

“…Not only is the description too highdimensional to be practical (e.g., any grasping action of a textile would alter its shape), but the type of data observed for the object is almost impossible to obtain through sensorization. State-of-the-art solutions, to handle this complexity, describe garments lying on a table using a polygonal shape (Doumanoglou et al, 2016), singular patches like corners or wrinkles (Kapusta et al, 2019), or cloth parts like collars and hemlines (Ramisa et al, 2016). Other approaches model the interaction focusing primarily on hand trajectories and grasping points (Corona et al, 2018;Zhang and Demiris, 2020).…”

Fast geometry-based computation of grasping points on three-dimensional point clouds

“…These are usually grasped by following different strategies, such as detecting the points which the gripper should pinch. 43 In addition, their dynamics require further supervision of shape deformation after the first grasping contact. 44 Nevertheless, our proposed grasping points can be used as an initial pair of contacts for this type of objects.…”

Exploiting Symmetries in Reinforcement Learning of Bimanual Robotic Tasks

“…R wrinkle penalizes the outcome if the towel presents too many wrinkles after the folding (the mean gradient of the depth is used as the wrinkleness indicator). It is computed using the code available from [18].…”