International audienceThe contribution of this work is to show that real-time nonlinear model predictive control (NMPC) can be implemented on position controlled humanoid robots. Following the idea of " walking without thinking " , we propose a walking pattern generator that takes into account simultaneously the position and orientation of the feet. A requirement for an application in real-world scenarios is the avoidance of obstacles. Therefore the paper shows an extension of the pattern generator that directly considers the avoidance of convex obstacles. The algorithm uses the whole-body dynamics to correct the center of mass trajectory of the underlying simplified model. The pattern generator runs in real-time on the embedded hardware of the humanoid robot HRP-2 and experiments demonstrate the increase in performance with the correction

Multi-contact motion generation is an important problem in humanoid robotics because it generalizes bipedal locomotion and thus expands the functional range of humanoid robots. In this paper, we propose a complete solution to compute a fully-dynamic multi-contact motion of a humanoid robot. We decompose the motion generation by computing first a dynamically-consistent trajectory of the center of mass of the robot and finding then the whole-body movement following this trajectory. A simplified dynamic model of the humanoid is used to find optimal contact forces as well as a kinematic feasible center-of-mass trajectory from a predefined series of contacts. We demonstrate the capabilities of the approach by making the real humanoid robot platform HRP-2 climb stairs with the use of a handrail. The experimental study also shows that utilization of the handrail lowers the power consumption of the robot by 25% compared to a motion, where only the feet are used.

Abstract-Learning-based approaches are suitable for the control of systems with unknown dynamics. However, learning from scratch involves many trials with exploratory actions until a good control policy is discovered. Real robots usually cannot withstand the exploratory actions and suffer damage. This problem can be circumvented by combining learning with model-based control. In this article, we employ a nominal model-predictive controller that is impeded by the presence of an unknown modelplant mismatch. To compensate for the mismatch, we propose two approaches of combining reinforcement learning with the nominal controller. The first approach learns a compensatory control action which minimizes the same performance measure as is minimized by the nominal controller. The second approach learns a compensatory signal from a difference of a transition predicted by the internal model and an actual transition. We compare the approaches on a robot attached to the ground and performing a setpoint reaching task in simulations. We implement the better approach on the real robot and demonstrate successful learning results.

Model-free reinforcement learning and nonlinear model predictive control are two different approaches for controlling a dynamic system in an optimal way according to a prescribed cost function. Reinforcement learning acquires a control policy through exploratory interaction with the system, while nonlinear model predictive control exploits an explicitly given mathematical model of the system. In this article, we provide a comprehensive comparison of the performance of reinforcement learning and nonlinear model predictive control for an ideal system as well as for a system with parametric and structural uncertainties. The comparison is based on two different criteria, namely the similarity of trajectories and the resulting rewards. The evaluation of both methods is performed on a standard benchmark problem: a cart-pendulum swing-up and balance task. We first find suitable mathematical formulations and discuss the effect of the differences in the problem formulations. Then, we investigate the robustness of reinforcement learning and nonlinear model predictive control against uncertainties. The results demonstrate that nonlinear model predictive control has advantages over reinforcement learning if uncertainties can be eliminated through identification of the system parameters. Otherwise, there exists a break-even point after which model-free reinforcement learning performs better than nonlinear model predictive control with an inaccurate model. These findings suggest that benefits can be obtained by combining these methods for real systems being subject to such uncertainties. In the future, we plan to develop a hybrid controller and evaluate its performance on a real seven-degree-of-freedom walking robot.

No abstract