Learning Postural Synergies for Categorical Grasping Through Shape Space Registration

Rodríguez, Diego; Guardo, Antonio Di; Frisoli, Antonio; Behnke, Sven

doi:10.1109/humanoids.2018.8625032

Cited by 10 publications

(6 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…of a deformation field, is found by searching in the shape space in a gradient descent manner, such that the deformed model matches the observation best according to a cost function. This function includes the point distances to the closest points, a local rigid transformation to account for global misalignment and a regularization term for the latent variables [19]. One advantage of using dense deformation fields is that points which do not belong to the canonical model can be deformed even after the registration has finished.…”

Section: A Non-rigid (Shape Space) Registrationmentioning

confidence: 99%

Autonomous Bimanual Functional Regrasping of Novel Object Class Instances

Pavlichenko

Rodríguez

Lenz

et al. 2019

2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids)

Self Cite

View full text Add to dashboard Cite

In human-made scenarios, robots need to be able to fully operate objects in their surroundings, i.e., objects are required to be functionally grasped rather than only picked. This imposes very strict constraints on the object pose such that a direct grasp can be performed. Inspired by the anthropomorphic nature of humanoid robots, we propose an approach that first grasps an object with one hand, obtaining full control over its pose, and performs the functional grasp with the second hand subsequently. Thus, we develop a fully autonomous pipeline for dual-arm functional regrasping of novel familiar objects, i.e., objects never seen before that belong to a known object category, e.g., spray bottles. This process involves semantic segmentation, object pose estimation, non-rigid mesh registration, grasp sampling, handover pose generation and in-hand pose refinement. The latter is used to compensate for the unpredictable object movement during the first grasp. The approach is applied to a human-like upper body. To the best knowledge of the authors, this is the first system that exhibits autonomous bimanual functional regrasping capabilities. We demonstrate that our system yields reliable success rates and can be applied on-line to real-world tasks using only one off-the-shelf RGB-D sensor.

show abstract

Section: A Non-rigid (Shape Space) Registrationmentioning

confidence: 99%

Autonomous Bimanual Functional Regrasping of Novel Object Class Instances

Pavlichenko

Rodríguez

Lenz

et al. 2019

2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids)

Self Cite

View full text Add to dashboard Cite

show abstract

“…To this end, we prepared a 3D mesh dataset of 39 objects: 13 of each category, where ten objects are for training and the remaining three objects are used for testing. The dataset was composed of meshes from [32] and of meshes available online 1 . We make the dataset available online 2 .…”

Section: A Setupmentioning

confidence: 99%

KittingBot: A Mobile Manipulation Robot for Collaborative Kitting in Automotive Logistics

Pavlichenko

García

Koo

et al. 2018

Intelligent Autonomous Systems 15

View full text Add to dashboard Cite

Individualized manufacturing of cars requires kitting: the collection of individual sets of part variants for each car. This challenging logistic task is frequently performed manually by warehouseman. We propose a mobile manipulation robotic system for autonomous kitting, building on the Kuka Miiwa platform which consists of an omnidirectional base, a 7 DoF collaborative iiwa manipulator, cameras, and distance sensors. Software modules for detection and pose estimation of transport boxes, part segmentation in these containers, recognition of part variants, grasp generation, and arm trajectory optimization have been developed and integrated. Our system is designed for collaborative kitting, i.e. some parts are collected by warehouseman while other parts are picked by the robot. To address safe human-robot collaboration, fast arm trajectory replanning considering previously unforeseen obstacles is realized. The developed system was evaluated in the European Robotics Challenge 2, where the Miiwa robot demonstrated autonomous kitting, part variant recognition, and avoidance of unforeseen obstacles.

show abstract

“…It is challenging to generate the grasp configuration for a new task by using postural synergy. This problem can be solved by inferring the synergy coordinates of objects from basic shape descriptions of them (Rodriguez and Behnke, 2018 ; Rodriguez et al, 2018 ).…”

Section: Introductionmentioning

confidence: 99%

Learning Grasp Configuration Through Object-Specific Hand Primitives for Posture Planning of Anthropomorphic Hands

et al. 2021

View full text Add to dashboard Cite

The proposal of postural synergy theory has provided a new approach to solve the problem of controlling anthropomorphic hands with multiple degrees of freedom. However, generating the grasp configuration for new tasks in this context remains challenging. This study proposes a method to learn grasp configuration according to the shape of the object by using postural synergy theory. By referring to past research, an experimental paradigm is first designed that enables the grasping of 50 typical objects in grasping and operational tasks. The angles of the finger joints of 10 subjects were then recorded when performing these tasks. Following this, four hand primitives were extracted by using principal component analysis, and a low-dimensional synergy subspace was established. The problem of planning the trajectories of the joints was thus transformed into that of determining the synergy input for trajectory planning in low-dimensional space. The average synergy inputs for the trajectories of each task were obtained through the Gaussian mixture regression, and several Gaussian processes were trained to infer the inputs trajectories of a given shape descriptor for similar tasks. Finally, the feasibility of the proposed method was verified by simulations involving the generation of grasp configurations for a prosthetic hand control. The error in the reconstructed posture was compared with those obtained by using postural synergies in past work. The results show that the proposed method can realize movements similar to those of the human hand during grasping actions, and its range of use can be extended from simple grasping tasks to complex operational tasks.

show abstract

Learning Postural Synergies for Categorical Grasping Through Shape Space Registration

Cited by 10 publications

References 30 publications

Autonomous Bimanual Functional Regrasping of Novel Object Class Instances

Autonomous Bimanual Functional Regrasping of Novel Object Class Instances

KittingBot: A Mobile Manipulation Robot for Collaborative Kitting in Automotive Logistics

Learning Grasp Configuration Through Object-Specific Hand Primitives for Posture Planning of Anthropomorphic Hands

Contact Info

Product

Resources

About