Nonparametric representation of an approximated Poincaré map for learning biped locomotion

Morimoto, Jun; Atkeson, Christopher G.

doi:10.1007/s10514-009-9133-z

Cited by 30 publications

(10 citation statements)

References 35 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this field, learning techniques are mostly applied to improve walking policies that were derived from simulations [7] or observed from human walking [8]. Furthermore, due to the complexity of biped locomotion, every walking robot is equipped with many sensors to determine its stance and to allow closed-loop control.…”

Section: Related Workmentioning

confidence: 99%

Collective motion pattern scaling for improved open-loop off-road navigation

Hoeller

Röhling

Schulz

2013

2013 13th International Conference on Autonomous Robot Systems

View full text Add to dashboard Cite

This paper presents an adaptive navigation system which is able to steer an electronically controlled ground vehicle to given destinations while it adjusts to changing surface conditions. The approach is designed for vehicles without a velocity controlled drive-train, making it especially useful for typical remote-controlled vehicles without upgraded motor controllers. The vehicle is controlled by sets of commands, each set representing a specific maneuver. These sets are combined to form trajectories towards a given destination. While one of these sets of commands is executed the vehicle's movement is measured to refine the geometry of all maneuvers. A scaling vector is derived from the changes in dimensions of the bounding boxes of the assumed and the actual path, which is then used to collectively update all known maneuvers. This enables the approach to quickly adapt to surface alterations. We tested our approach using a 300 kg Explosive Ordnance Disposal (EOD) robot in an outdoor environment. The experiments confirmed that the Collective Motion Pattern Scaling significantly increases the adaptation performance compared to an approach without collective scaling

show abstract

Section: Related Workmentioning

confidence: 99%

Collective motion pattern scaling for improved open-loop off-road navigation

Hoeller

Röhling

Schulz

2013

2013 13th International Conference on Autonomous Robot Systems

View full text Add to dashboard Cite

show abstract

“…However, in more dynamic control problems, even without considering reward functions at the moment, the robot has to consider a highdimensional state space and estimate a hidden variable online. In such situations, the complex external dynamics need to be reduced using sophisticated methods (Stephens, 2007;Morimoto & Atkeson, 2009;Kajita, Kanehiro, Kaneko, Fujiwara, & Yokoi, 2003). To apply these techniques developed in robotics to the RL dynamics, it is convenient for RL robots to have a modular architecture in which dynamics and reward can be addressed independently.…”

Section: Introductionmentioning

confidence: 99%

MOSAIC for Multiple-Reward Environments

et al. 2012

View full text Add to dashboard Cite

Reinforcement learning (RL) can provide a basic framework for autonomous robots to learn to control and maximize future cumulative rewards in complex environments. To achieve high performance, RL controllers must consider the complex external dynamics for movements and task (reward function) and optimize control commands. For example, a robot playing tennis and squash needs to cope with the different dynamics of a tennis or squash racket and such dynamic environmental factors as the wind. In addition, this robot has to tailor its tactics simultaneously under the rules of either game. This double complexity of the external dynamics and reward function sometimes becomes more complex when both the multiple dynamics and multiple reward functions switch implicitly, as in the situation of a real (multi-agent) game of tennis where one player cannot observe the intention of her opponents or her partner. The robot must consider its opponent's and its partner's unobservable behavioral goals (reward function). In this article, we address how an RL agent should be designed to handle such double complexity of dynamics and reward. We have previously proposed modular selection and identification for control (MOSAIC) to cope with nonstationary dynamics where appropriate controllers are selected and learned among many candidates based on the error of its paired dynamics predictor: the forward model. Here we extend this framework for RL and propose MOSAIC-MR architecture. It resembles MOSAIC in spirit and selects and learns an appropriate RL controller based on the RL controller's TD error using the errors of the dynamics (the forward model) and the reward predictors. Furthermore, unlike other MOSAIC variants for RL, RL controllers are not a priori paired with the fixed predictors of dynamics and rewards. The simulation results demonstrate that MOSAIC-MR outperforms other counterparts because of this flexible association ability among RL controllers, forward models, and reward predictors.

show abstract

“…However, due to the nonlinear dynamic property of the coupled oscillator system composed of the CPG controller and the robot, it is rather difficult to analytically design the biped trajectory to satisfy the requirements of a target walking pattern. Therefore, using a nonlinear optimization method to improve the walking trajectory is reasonable [6], [7]. To optimize the walking trajectory, a model-free optimal control method is preferable because precisely modeling the ground contact is difficult.…”

Section: Introductionmentioning

confidence: 99%

Phase-dependent trajectory optimization for CPG-based biped walking using path integral reinforcement learning

Sugimoto

Morimoto²

2011

2011 11th IEEE-RAS International Conference on Humanoid Robots

Self Cite

View full text Add to dashboard Cite

In this study, we introduce a phase-dependent trajectory optimization method for Central Pattern Generator (CPG)-based biped walking controllers. By exploiting the syn chronization property of the CPG controller, many legged loco motion studies have shown that the CPG-based walking controller is robust against external perturbations and works well in real environments. However, due to the nonlinear dynamic property of the coupled oscillator system composed of the CPG controller and the robot, analytically designing the biped trajectory to satisfy the requirements of a target walking pattern is rather difficult.Therefore, using a nonlinear optimization method is reasonable to improve the walking trajectory. To optimize the walking trajectory, a model-free optimal control method is preferable because precise modeling of the ground contact is difficult. On the other hand, model-free trajectory optimization methods have been considered as quite computationally demanding approach.However, because of recent advances in the nonlinear trajectory optimization method, using the model-free optimization method is now a realistic approach for biped trajectory optimization. We use a path integral reinforcement learning method to improve the biped walking trajectory for CPG-based walking controllers.

show abstract

Nonparametric representation of an approximated Poincaré map for learning biped locomotion

Cited by 30 publications

References 35 publications

Collective motion pattern scaling for improved open-loop off-road navigation

Collective motion pattern scaling for improved open-loop off-road navigation

MOSAIC for Multiple-Reward Environments

Phase-dependent trajectory optimization for CPG-based biped walking using path integral reinforcement learning

Contact Info

Product

Resources

About