Adaptive whole-body manipulation in human-to-humanoid multi-contact motion retargeting

Abstract

Abstract-We propose a controller for loco-manipulation motion retargeting from a human to a humanoid robot. Using this controller, the robot can track complex motions and automatically adapt to elements in the environment that have different physical properties from those that were used to provide the human's reference motion. The multi-contact loco-manipulation problem is formulated as a multi-robot quadratic program (MRQP), which optimizes over the combined dynamics of the robot and any manipulated element in the environment. Our approach maintains a dynamic partition of the robot's tracking links into fixed support contact links, manipulation contact links, and contact-free tracking links. The three sets are repartitioned and re-instantiated as objectives or constraints in the MRQP when contact events occur in the human motion. We present various experiments (bag retrieval, door opening, box lifting) using human motion data from an Xsens inertial motion capture system. We show in dynamics simulation that the robot is able to perform difficult single-stance motions as well as multi-contact-stance motions (including hand supports), while adapting to environment elements of varying inertial properties.

Humanoid Whole-Body Movement Optimization from Retargeted Human Motions

“…In [1], the authors retarget human motion onto a humanoid robot using a QP controller and additionally taking into consideration higher-level constraints such as balance. Other works have achieved motion retargeting in other challenging scenarios, with multiple contacts [2], or heavy object manipulation [13]. Furthermore, demonstrations may be used to learn useful robot control policies.…”

“…It is assumed that the demonstrations from humans for a given activity are already close to minimize Eq. (13). Additionally, to increase the positive influence on the search for w , we add sets of high-level non-linear constraints such as Robot must be standing up at the end of trajectory execution; and the robot must reach a set of fixed targets in space during the trajectory.…”

Robust Real-Time Whole-Body Motion Retargeting from Human to Humanoid

“…Recently, multi-task whole-body QP controllers were applied to retargeting. Otani and Bouyarmane [12] retargeted onto the HRP-4 robot in full-body dynamics simulation a bag retrieval, a door opening and a box lifting task. Their method relied on a-priori knowledge of the contact events and could not be formulated on-line without an off-line pre-processing of the recorded human motion sequence.…”

Generating Assistive Humanoid Motions for Co-Manipulation Tasks with a Multi-Robot Quadratic Program Controller

“…Below is a complete list of the tasks used in the experiments, along with their relative weights: Contact change events were manually annotated in the recorded data, and handled via the method described in our prior work on motion retargeting [23]. In co-manipulation scenarios, we assume that the contact changes are simultaneous for the human and robot.…”