Constrained DMPs for Feasible Skill Learning on Humanoid Robots

Abstract

In the context of humanoid skill learning, movement primitives have gained much attention because of their compact representation and convenient combination with a myriad of optimization approaches. Among them, a well-known scheme is to use Dynamic Movement Primitives (DMPs) with reinforcement learning (RL) algorithms. While various remarkable results have been reported, skill learning with physical constraints has not been sufficiently investigated. For example, when RL is employed to optimize the robot joint trajectories, the exploration noise could drive the resulting trajectory out of the joint limits. In this paper, we focus on robot skill learning characterized by joint limit avoidance, by introducing the novel Constrained Dynamic Movement Primitives (CDMPs). By controlling a set of transformed states (called exogenous states) instead of the original DMPs states, CDMPs are capable of maintaining the joint trajectories within the safety limits. We validate CDMPs on the humanoid robot iCub, showing the applicability of our approach.

Learning to Sequence Multiple Tasks with Competing Constraints

“…More widely applicable results could be obtained by further investigating these aspects. Also, to make robots safely operate in uncluttered environments, it can be considered to incorporate obstacle and joint limit avoidance [19]. Another interesting extension would be to learn the activation function profile π in response to the surrounding environment.…”

“…We need to transform Cartesian trajectories into joint space in order to control the robot. To do so, the Jacobian-based inverse kinematics techniqueẋ = J(q)q is employed, where J ∈ R 3×d is the robot Jacobian matrix [19]. In general, the corresponding discrete implementation incorporating nullspace exploration is given by…”

Ultrasound-Guided Assistive Robots for Scoliosis Assessment with Optimization-based Control and Variable Impedance

“…In view of high complexity of the multi-input multi-output system in addition to several constraints that could emerge, such as control bounds, joint limits, kinematic constraints etc., we consider to formulate our control problem from an optimization perspective. Compared with analytic control law design, optimization-based control strategies exhibit great potential for customization towards different requirements [24] and better at explicitly handling constraints [25]. Specifically, our control method akin to a feedback linearization method is composed of two loops similar to [20].…”

Learning to Avoid Obstacles With Minimal Intervention Control

“…Moreover, given that trajectories will be modified by the potential field unpredictably during obstacle avoidance, there are possibilities that joint trajectory evolution could exceed the allowable range. To address this issue, we employ Constrained Dynamic Movement Primitives (CDMPs), recently developed by Duan et al ( 2018 ) so as to ensure joint trajectories are always bounded within the specified range.…”

“…It can be conceived that the modified trajectories are very susceptible to the strength of the potential field and a strong field could drive joint trajectories out of the allowable range. To cope with such issue, we will drive the robot's trajectories using our recently developed Constrained Dynamic Movement Primitives (CDMPs) (Duan et al, 2018 ). CDMPs are derived by parameterizing the original trajectory using the hyperbolic tangent function .…”