Visual grasp point localization, classification and state recognition in robotic manipulation of cloth: An overview

Abstract

Cloth manipulation by robots is gaining popularity among researchers because of its relevance, mainly (but not only) in domestic and assistive robotics. The required science and technologies begin to be ripe for the challenges posed by the manipulation of soft materials, and many contributions have appeared in the last years. This survey provides a systematic review of existing techniques for the basic perceptual tasks of grasp point localization, state estimation and classification of cloth items, from the perspective of their manipulation by robots. This choice is grounded on the fact that any manipulative action requires to instruct the robot where to grasp, and most garment handling activities depend on the correct recognition of the type to which the particular cloth item belongs and its state. The high inter-and intraclass variability of garments, the continuous nature of the possible deformations of cloth and the evident difficulties in predicting their localization and extension on the garment piece are challenges that have encouraged the researchers to provide a plethora of methods to confront such problems, with some promising results. The present review constitutes for the first time an effort in furnishing a structured framework of these works, with the aim of helping future contributors to gain both insight and perspective on the subject.

Grasping Objects Mixed With Towels

“…In the procedure of folding, corner points are often considered as the grasping positions [9]. For further information on grasping clothes, please refer to the survey in [23]. In summary, when grasping rigid objects, existing works have mainly focused on the generation of grasping poses on the objects.…”

Feature extraction of garments based on Gaussian mixture for autonomous robotic manipulation

“…In this case, these edges cannot be considered as a contour. So, performance of Canny algorithm is adequate provided background subtraction results are reasonably efficient [7]. Moore's algorithm gives a single connected contour having the largest connected area.…”

“…Here, the N-vertex polygonal contour 𝑃 obtained from Moore's algorithm is approximated into another polygonal contour 𝑄 with the minimum number of line segments 𝑀 such that the approximation error 𝐸(𝑝) is less than error bound (can be around 2%) [12]. 𝐸(𝑝) is defined as a Euclidean distance from the vertices of the curve 𝑃 to the approximated line segment of curve 𝑄. 𝐸(𝑃) = 𝑚𝑎𝑥 1≤𝑚≤𝑀 𝑑(𝑞 𝑚 , 𝑞 𝑚+1 ) (7) where 𝑑(𝑝 𝑖 , 𝑝 𝑗 ) = 𝑚𝑎𝑥 𝑖≤𝑘≤𝑗 𝑘; 𝑑(𝑝 𝑖 , 𝑝 𝑗 ) and 𝑞 𝑚 = 𝑝 𝑖 , 𝑞 𝑚+1 = 𝑝 𝑗…”

Fabric Classification Using a Finger-Shaped Tactile Sensor via Robotic Sliding

“…With the advancement in interactive control for robotics, tactile sensing is gaining attention in recent decades. Since robotic tasks with physical contacts are very likely to introduce visual occlusion, more studies using tactile sensing to perceive object shape/textures (Kaboli and Cheng, 2018 ; Kerr et al, 2018 ; Martinez-Hernandez et al, 2020 ; Fang et al, 2021 ) and executing dexterous manipulation (Jiménez, 2017 ; Belousov et al, 2019 ) are popping up. Results show that tactile sensing has great potential, especially in handling soft materials like fabrics considering its instant response to tiny variations of stimulus.…”