We propose a framework for combining vision and haptic information in human-robot joint actions. It consists of a hybrid controller that uses both visual servoing and impedance controllers. This can be applied to tasks that cannot be done with vision or haptic information alone. In this framework, the state of the task can be obtained from visual information while haptic information is crucial for safe physical interaction with the human partner. The approach is validated on the task of jointly carrying a flat surface (e.g. a table) and then preventing an object (e.g. a ball) on top from falling off. The results show that this task can be successfully achieved. Furthermore, the framework presented allows for a more collaborative setup, by imparting task knowledge to the robot as opposed to a passive follower.

Abstract-In this paper, we propose a control scheme that allows a humanoid robot to perform a transportation task jointly with a human partner. From the study of how human dyads achieve such a task, we have developed a control law for physical interaction that unifies standalone and collaborative (leader and follower) modes for trajectory-based tasks. We present it in the case of a linear impedance controller but it can be generalized to more complex impedances. Desired trajectories are decomposed into sequences of elementary motion primitives. We implemented this model with a Finite State Machine associated with a reactive pattern generator. First experiments conducted on a real HRP-2 humanoid robot assess the overall approach.

Abstract-In this paper, we propose a control scheme that allows a humanoid robot to perform a complex transportation scenario jointly with a human partner. At first, the robot guesses the human partner's intentions to proactively participate to the task. In a second phase, the human-robot dyad switches roles: the robot takes over the leadership of the task to complete the scenario. During this last phase, the robot is remotely controlled with a joystick. The scenario is realized on a real HRP-2 humanoid robot to assess the overall approach.