Using depth and appearance features for informed robot grasping of highly wrinkled clothes

Ramisa, Arnau; Alenyà, Guillem; Moreno-Noguer, Francesc; Torras, Carme

doi:10.1109/icra.2012.6225045

Cited by 116 publications

(83 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the hanged shirt the sleeves are identifiable as well. Previous works have shown that this 3D structure can be used to identify wrinkles [56] and also the collar structure, using computer vision algorithms [57].…”

Section: Object-related Tasksmentioning

confidence: 99%

ToF cameras for active vision in robotics

Alenyà

Foix

Torras

2014

Sensors and Actuators A: Physical

View full text Add to dashboard Cite

ToF cameras are now a mature technology that is widely being adopted to provide sensory input to robotic applications. Depending on the nature of the objects to be perceived and the viewing distance, we distinguish two groups of applications: those requiring to capture the whole scene and those centered on an object. It will be demonstrated that it is in this last group of applications, in which the robot has to locate and possibly manipulate an object, where the distinctive characteristics of ToF cameras can be better exploited.After presenting the physical sensor features and the calibration requirements of such cameras, we review some representative works highlighting for each one which of the distinctive ToF characteristics have been more essential. Even if at low resolution, the acquisition of 3D images at frame-rate is one of the most important features, as it enables quick background/foreground segmentation. A common use is in combination with classical color cameras. We present three developed applications, using a mobile robot and a robotic arm, to exemplify with real images some of the stated advantages.

show abstract

Section: Object-related Tasksmentioning

confidence: 99%

ToF cameras for active vision in robotics

Alenyà

Foix

Torras

2014

Sensors and Actuators A: Physical

View full text Add to dashboard Cite

show abstract

“…In that regard, Ramisa et al [8] propose a supervised learning approach for grasping highly wrinkled garments. In Ramisa's grasp point detection method, manually segmented regions are used to train logistic regression and χ 2 SVM classifiers using bag-of-words descriptors based on SIFT [10] and Geodesic-Depth Histogram features [11].…”

Section: Literature Reviewmentioning

confidence: 99%

“…Research on perception and manipulation tasks for robotic laundry systems consists of: isolating and classifying clothes from a laundry heap [8,5,9], finding a tractable state of the clothes in order to interact and manipulate them [3] and then folding the cloth [1,2,5]. In that regard, Ramisa et al [8] propose a supervised learning approach for grasping highly wrinkled garments.…”

Section: Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

A Precise Method for Cloth Configuration Parsing Applied to Single-Arm Flattening

Sun

Aragon-Camarasa

Khan

et al. 2016

International Journal of Advanced Robotic Systems

View full text Add to dashboard Cite

In this paper, we investigate the contribution that visual perception affords to a robotic manipulation task in which a crumpled garment is flattened by eliminating visually detected wrinkles. In order to explore and validate visually guided clothing manipulation in a repeatable and controlled environment, we have developed a hand-eye interactive virtual robot manipulation system that incorporates a clothing simulator to close the effector-garment-visual sensing interaction loop. We present the technical details and compare the performance of two different methods for detecting, representing and interpreting wrinkles within clothing surfaces captured in high-resolution depth maps. The first method we present relies upon a clustering-based method for localizing and parametrizing wrinkles, while the second method adopts a more advanced geometrybased approach in which shape-topology analysis underpins the identification of the cloth configuration (i.e., maps wrinkles). Having interpreted the state of the cloth configuration by means of either of these methods, a heuristicbased flattening strategy is then executed to infer the appropriate forces, their directions and gripper contact locations that must be applied to the cloth in order to flatten the perceived wrinkles. A greedy approach, which attempts to flatten the largest detected wrinkle for each perception-iteration cycle, has been successfully adopted in this work. We present the results of our heuristic-based flattening methodology which relies upon clustering-based and geometry-based features respectively. Our experiments indicate that geometry-based features have the potential to provide a greater degree of clothing configuration understanding and, as a consequence, improve flattening performance. The results of experiments using a real robot (as opposed to simulated robot) also confirm our proposition that a more effective visual perception system can advance the performance of cloth manipulation.

show abstract

“…This paper is an extension of Ramisa et al [4,5] where a preliminary version of the part detection pipeline and the FINDDD descriptor were presented, respectively. The current paper combines and extends both previous contributions in a pipeline for garment manipulation, and it provides a more thorough mathematical description of the methods used, as well as a more extensive set of evaluations (e.g.…”

Section: Introductionmentioning

confidence: 99%

A 3D descriptor to detect task-oriented grasping points in clothing

Ramisa

Alenyà

Moreno-Noguer

et al. 2016

Pattern Recognition

Self Cite

View full text Add to dashboard Cite

Manipulating textile objects with a robot is a challenging task, especially because the garment perception is difficult due to the endless configurations it can adopt, coupled with a large variety of colors and designs. Most current approaches follow a multiple re-grasp strategy, in which clothes are sequentially grasped from different points until one of them yields a recognizable configuration. In this work we propose a method that combines 3D and appearance information to directly select a suitable grasping point for the task at hand, which in our case consists of hanging a shirt or a polo shirt from a hook. Our method follows a coarse-to-fine approach in which, first, the collar of the garment is detected and, next, a grasping point on the lapel is chosen using a novel 3D descriptor.In contrast to current 3D descriptors, ours can run in real time, even when it needs to be densely computed over the input image. Our central idea is to take advantage of the structured nature of range images that most depth sensors provide and, by exploiting integral imaging, achieve speed-ups of two orders of magnitude with respect to competing approaches, while maintaining performance. This makes it especially adequate for robotic applications as we thoroughly demonstrate in the experimental section.

show abstract

Using depth and appearance features for informed robot grasping of highly wrinkled clothes

Cited by 116 publications

References 14 publications

ToF cameras for active vision in robotics

ToF cameras for active vision in robotics

A Precise Method for Cloth Configuration Parsing Applied to Single-Arm Flattening

A 3D descriptor to detect task-oriented grasping points in clothing

Contact Info

Product

Resources

About