Transferring impedance control strategies between heterogeneous systems via apprenticeship learning

Abstract

Abstract-We present a novel method for designing controllers for robots with variable impedance actuators. We take an imitation learning approach, whereby we learn impedance modulation strategies from observations of behaviour (for example, that of humans) and transfer these to a robotic plant with very different actuators and dynamics. In contrast to previous approaches where impedance characteristics are directly imitated, our method uses task performance as the metric of imitation, ensuring that the learnt controllers are directly optimised for the hardware of the imitator. As a key ingredient, we use apprenticeship learning to model the optimisation criteria underlying observed behaviour, in order to frame a correspondent optimal control problem for the imitator. We then apply local optimal feedback control techniques to find an appropriate impedance modulation strategy under the imitator's dynamics. We test our approach on systems of varying complexity, including a novel, antagonistic series elastic actuator and a biologically realistic two-joint, six-muscle model of the human arm.

Exploiting variable physical damping in rapid movement tasks

“…On the right hand side of the above equation there are the torques generated by the driver motor 2 according to (7). During the experiments the driving motor is supplied with a constant voltage V d = 24V while the duty cycle of the PWM signal u 3 ∈ {0, 0.1, ...1} was changed by ten percent at every 5 s. At each of these stages, the system quickly converged to a steady rotation speed (θ = 0).…”

“…For example, optimal control approaches have been shown to be highly effective in exploiting the elastic properties of variable stiffness actuators in explosive tasks such as throwing and hitting [4], [7] as well as in periodic tasks [8], [9], [10]. In addition, stochastic optimal control with model adaptation has also been exploited in order to cope with model uncertainty and perturbations [11], [12], [13].…”

Online learning of varying stiffness through physical human-robot interaction

“…Thus, the resulting policies are tradeoffs between task performance and energy consumption. [3] further uses inverse optimal control to infer a task-based costfunction in order to transfer variable impedance policies between different systems. In [4] analytical solutions to optimal control of variable stiffness for maximizing link velocity is reported.…”

“…A recent trend in PbD tackles the problem of teaching force-based control policies [8], [9]. In this work, we follow a similar approach, in that the robot is implicitly 3 taught the forces that it should exert in response to perturbations. This is in stark contrast to classical approaches in PbD that usually relied on kinematic information only.…”

“…Learning variable impedance control policies has been formulated as an optimal control (OC) problem in [2] and [3]. These works specify the task constraints as a cost function and optimize the control actions subject to the dynamics of the robot.…”