Minimax differential dynamic programming: application to a biped walking robot

Morimioto, J.; Zeglin, Garth; Atkeson, Christopher G.

doi:10.1109/iros.2003.1248926

Cited by 140 publications

(123 citation statements)

References 7 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…In the robotics community, the combination of DDP with MPC is a popular approach, providing a practical compromise between stability, non-linearity and efficient computation and has been succesfully applied to robot walking and manipulation [29,43] and aerobatic helicopter flight [1].…”

Section: Model Predictive Controlmentioning

confidence: 99%

Adaptive Importance Sampling for Control and Inference

Kappen

Ruiz

2016

J Stat Phys

View full text Add to dashboard Cite

Path integral (PI) control problems are a restricted class of non-linear control problems that can be solved formally as a Feynman-Kac PI and can be estimated using Monte Carlo sampling. In this contribution we review PI control theory in the finite horizon case. We subsequently focus on the problem how to compute and represent control solutions. We review the most commonly used methods in robotics and control. Within the PI theory, the question of how to compute becomes the question of importance sampling. Efficient importance samplers are state feedback controllers and the use of these requires an efficient representation. Learning and representing effective state-feedback controllers for non-linear stochastic control problems is a very challenging, and largely unsolved, problem. We show how to learn and represent such controllers using ideas from the cross entropy method. We derive a gradient descent method that allows to learn feed-back controllers using an arbitrary parametrisation. We refer to this method as the path integral cross entropy method or PICE. We illustrate this method for some simple examples. The PI control methods can be used to estimate the posterior distribution in latent state models. In neuroscience these problems arise when estimating connectivity from neural recording data using EM. We demonstrate the PI control method as an accurate alternative to particle filtering.

show abstract

Section: Model Predictive Controlmentioning

confidence: 99%

Adaptive Importance Sampling for Control and Inference

Kappen

Ruiz

2016

J Stat Phys

View full text Add to dashboard Cite

show abstract

“…The main goal of this paper is to discuss which policy gradient methods are applicable to robotics and which issues matter, while also introducing some new policy gradient learning algorithms that seem to have superior 1 Note that there has been earlier work by the control community, see e.g., Dyer and McReynolds (1970), Hasdorff (1976) and Jacobson and Mayne (1970), which is based on exact analytical models. Extensions based on learned, approximate models originated in the literature on optimizing government decision policies, see Werbos (1979), and have also been applied in control (Atkeson, 1994;Morimoto & Atkeson, 2003). In this paper, we limit ourselves to model-free approaches as the most general framework, while future work will address specialized extensions to model-based learning.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning of motor skills with policy gradients

2008

View full text Add to dashboard Cite

a b s t r a c tAutonomous learning is one of the hallmarks of human and animal behavior, and understanding the principles of learning will be crucial in order to achieve true autonomy in advanced machines like humanoid robots. In this paper, we examine learning of complex motor skills with human-like limbs. While supervised learning can offer useful tools for bootstrapping behavior, e.g., by learning from demonstration, it is only reinforcement learning that offers a general approach to the final trial-and-error improvement that is needed by each individual acquiring a skill. Neither neurobiological nor machine learning studies have, so far, offered compelling results on how reinforcement learning can be scaled to the high-dimensional continuous state and action spaces of humans or humanoids. Here, we combine two recent research developments on learning motor control in order to achieve this scaling. First, we interpret the idea of modular motor control by means of motor primitives as a suitable way to generate parameterized control policies for reinforcement learning. Second, we combine motor primitives with the theory of stochastic policy gradient learning, which currently seems to be the only feasible framework for reinforcement learning for humanoids. We evaluate different policy gradient methods with a focus on their applicability to parameterized motor primitives. We compare these algorithms in the context of motor primitive learning, and show that our most modern algorithm, the Episodic Natural Actor-Critic outperforms previous algorithms by at least an order of magnitude. We demonstrate the efficiency of this reinforcement learning method in the application of learning to hit a baseball with an anthropomorphic robot arm.

show abstract

“…Both posture control and biped locomotion are challenging research topics for a human-like and autonomous humanoid robot. Our exploration focused on biologically inspired control algorithms for locomotion using three different humanoid robots (DB-chan [72,76,77], Fujitsu Automation HOAP-2 [48] and CB-i [78]) as well as a small humanoid robot provided by the SONY Corp. [79].…”

Section: Creating a Brain By Brain-motivated Humanoid-motor-learning mentioning

confidence: 99%

Creating the brain and interacting with the brain: an integrated approach to understanding the brain

Morimoto¹,

Kawato

2015

J. R. Soc. Interface.

View full text Add to dashboard Cite

In the past two decades, brain science and robotics have made gigantic advances in their own fields, and their interactions have generated several interdisciplinary research fields. First, in the 'understanding the brain by creating the brain' approach, computational neuroscience models have been applied to many robotics problems. Second, such brain-motivated fields as cognitive robotics and developmental robotics have emerged as interdisciplinary areas among robotics, neuroscience and cognitive science with special emphasis on humanoid robots. Third, in brain-machine interface research, a brain and a robot are mutually connected within a closed loop. In this paper, we review the theoretical backgrounds of these three interdisciplinary fields and their recent progress. Then, we introduce recent efforts to reintegrate these research fields into a coherent perspective and propose a new direction that integrates brain science and robotics where the decoding of information from the brain, robot control based on the decoded information and multimodal feedback to the brain from the robot are carried out in real time and in a closed loop.

show abstract

Minimax differential dynamic programming: application to a biped walking robot

Abstract: Abstract

Cited by 140 publications

References 7 publications

Adaptive Importance Sampling for Control and Inference

Adaptive Importance Sampling for Control and Inference

Reinforcement learning of motor skills with policy gradients

Creating the brain and interacting with the brain: an integrated approach to understanding the brain

Contact Info

Product

Resources

About