The KIT Bimanual Manipulation Dataset

Krebs, Franziska; Meixner, Andre; Patzer, Isabel; Asfour, Tamim

doi:10.1109/humanoids47582.2021.9555788

Cited by 41 publications

(23 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As the considered motions mostly involve the movement of a single arm, the influence of the remaining DoFs of the body on the planned geodesic synergies remains limited. However, many human motions involve the coordination of multiple joints throughout the whole body, e.g., for bimanual manipulation or loco-manipulation tasks [34]. As discussed by Neilson et al [16], whole-body motions may not be solely explained by single geodesic synergies, but instead by several synergies activated simultaneously.…”

Section: Discussionmentioning

confidence: 99%

A Riemannian Take on Human Motion Analysis and Retargeting

Klein¹,

Jaquier²,

Meixner³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Dynamic motions of humans and robots are widely driven by posture-dependent nonlinear interactions between their degrees of freedom. However, these dynamical effects remain mostly overlooked when studying the mechanisms of human movement generation. Inspired by recent works, we hypothesize that human motions are planned as sequences of geodesic synergies, and thus correspond to coordinated joint movements achieved with piecewise minimum energy. The underlying computational model is built on Riemannian geometry to account for the inertial characteristics of the body. Through the analysis of various human arm motions, we find that our model segments motions into geodesic synergies, and successfully predicts observed arm postures, hand trajectories, as well as their respective velocity profiles. Moreover, we show that our analysis can further be exploited to transfer arm motions to robots by reproducing individual human synergies as geodesic paths in the robot configuration space.

show abstract

Section: Discussionmentioning

confidence: 99%

A Riemannian Take on Human Motion Analysis and Retargeting

Klein¹,

Jaquier²,

Meixner³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To achieve good performance and generalization capabilities, supervised learning algorithms require quality annotated and abundant training data. For instance, recent work [27], [28], [29] presents novel multimodal datasets of humans grasping or manipulating objects which are remarkably valuable for benchmarking and performance analysis purposes. However, they are acquired with highly sensorized, extremely controlled setups.…”

Section: Related Workmentioning

confidence: 99%

Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis

Federico¹,

Maiettini²,

Pasquale³

et al. 2022

Preprint

View full text Add to dashboard Cite

We consider the task of object grasping with a prosthetic hand capable of multiple grasp types. In this setting, communicating the intended grasp type often requires a high user cognitive load which can be reduced adopting shared autonomy frameworks. Among these, so-called eye-inhand systems automatically control the hand aperture and preshaping before the grasp, based on visual input coming from a camera on the wrist. In this work, we present an eye-inhand learning-based approach for hand pre-shape classification from RGB sequences. In order to reduce the need for tedious data collection sessions for training the system, we devise a pipeline for rendering synthetic visual sequences of hand trajectories for the purpose. We tackle the peculiarity of the eye-in-hand setting by means of a model for the human arm trajectories, with domain randomization over relevant visual elements. We develop a sensorized setup to acquire real human grasping sequences for benchmarking and show that, compared on practical use cases, models trained with our synthetic dataset achieve better generalization performance than models trained on real data. We finally integrate our model on the Hannes prosthetic hand and show its practical effectiveness. Our code, real and synthetic datasets will be released upon acceptance.

show abstract

“…We aim at evaluating our approach to sequence and blend skills based on human demonstrations, i.e., on data for which no ground truth is easily available. To do so, we consider a bimanual sweeping task from the KIT motion database [31], [32], in which a human transfers cucumber slices from a cutting board to a bowl. At the beginning of the demonstrations, a subject stands in front of a table.…”

Section: Bimanual Sweeping Task Learned From Human Datamentioning

confidence: 99%

Learning to Sequence and Blend Robot Skills via Differentiable Optimization

Jaquier¹,

Zhou²,

Starke³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In contrast to humans and animals who naturally execute seamless motions, learning and smoothly executing sequences of actions remains a challenge in robotics. This paper introduces a novel skill-agnostic framework that learns to sequence and blend skills based on differentiable optimization. Our approach encodes sequences of previously-defined skills as quadratic programs (QP), whose parameters determine the relative importance of skills along the task. Seamless skill sequences are then learned from demonstrations by exploiting differentiable optimization layers and a tailored loss formulated from the QP optimality conditions. Via the use of differentiable optimization, our work offers novel perspectives on multitask control. We validate our approach in a pick-and-place scenario with planar robots, a pouring experiment with a real humanoid robot, and a bimanual sweeping task with a human model.

show abstract

The KIT Bimanual Manipulation Dataset

Cited by 41 publications

References 40 publications

A Riemannian Take on Human Motion Analysis and Retargeting

A Riemannian Take on Human Motion Analysis and Retargeting

Grasp Pre-shape Selection by Synthetic Training: Eye-in-hand Shared Control on the Hannes Prosthesis

Learning to Sequence and Blend Robot Skills via Differentiable Optimization

Contact Info

Product

Resources

About