Ensemble-CIO: Full-body dynamic motion planning that transfers to physical humanoids

Mordatch, Igor; Lowrey, Kendall; Todorov, Emanuel

doi:10.1109/iros.2015.7354126

Cited by 126 publications

(109 citation statements)

References 10 publications

Supporting

Mentioning

109

Contrasting

Order By: Relevance

“…The concept of UP is similar to Mordatch et al [21] who optimized the motion trajectory for an ensemble of dynamic models perturbed from the assumed one. Their method shows high success rate when tracking a real-world reference trajectory.…”

Section: B Transfer Learning In Reinforcement Learningmentioning

confidence: 99%

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Yu¹,

Tan²,

Liu³

et al. 2017

Robotics: Science and Systems XIII

185

148

View full text Add to dashboard Cite

Abstract-We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters µ. The value of µ from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.

show abstract

Section: B Transfer Learning In Reinforcement Learningmentioning

confidence: 99%

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Yu¹,

Tan²,

Liu³

et al. 2017

Robotics: Science and Systems XIII

185

148

View full text Add to dashboard Cite

show abstract

“…Mordatch et al [26] developed an ensemble trajectory optimization method that aims to reduce the expected cost under random model parameters while reducing trajectory variation under PD control. Lou and Hauser [18] combined robust motion planning with model estimation to optimize robust motions involving contact changes.…”

Section: Related Workmentioning

confidence: 99%

“…This paper builds on previous work on robust motion planning based on direct trajectory optimization [26,3] and differential dynamic programming (DDP) [27,6,29]. Robust motion planning algorithms often differ in the precise notion of robustness that they seek to optimize.…”

Section: Introductionmentioning

confidence: 99%

DIRTREL: Robust Trajectory Optimization with Ellipsoidal Disturbances and LQR Feedback

Manchester

Kuindersma

2017

Robotics: Science and Systems XIII

View full text Add to dashboard Cite

Abstract-Many critical robotics applications require robustness to disturbances arising from unplanned forces, state uncertainty, and model errors. Motion planning algorithms that explicitly reason about robustness require a coupling of trajectory optimization and feedback design, where the system's closedloop response to bounded disturbances is optimized. Due to the often-heavy computational demands of solving such problems, the practical application of robust trajectory optimization in robotics has so far been limited. We derive a tractable robust optimization algorithm that combines direct transcription with linear-quadratic control design to reason about closed-loop responses to disturbances. In the case of ellipsoidal disturbance sets, the state and input deviations along a nominal trajectory can be computed locally in closed form, thus allowing for fast evaluations of robust cost and constraint functions. The resulting algorithm, called DIRTREL, is an extension of classical direct transcription that demonstrably improves tracking performance over non-robust formulations while incurring only a modest increase in computational cost. We evaluate the algorithm in several simulated robot control tasks.

show abstract

“…However, to try to obtain a response that works reasonably well for all the experiments in different initial conditions, we have decided to modify slightly the cost function of the basic system identification problem to perform the identification on several tests at once. This idea was taken from [Mortdatch et al, 2015] where a similar set-up was used to synthesize trajectories of a biped robot with an unreliable model. Following this idea the cost function used in this work can be written as follows:…”

Section: Sandmentioning

confidence: 99%

Parameter identification and modeling of contact properties for robotic applications

Reggeti¹,

Carlos²

View full text Add to dashboard Cite

AcknowledgmentIn first place I would like to thank the Centre for Automation and Robotics UPM-CSIC in which this thesis was carried out, and the Spanish National Research Council for funding my research.I am especially grateful to Dr. Elena Garcia under whom direction and advise this research has been carried out.I would like to express my gratitude towards my coworkers (Daniel, Manuel, Fernando, Luis, etc.) for they were very supportive and a great aid at some moments.Finally, I would like to thank Mariana for her support and comprehension, and my parents and family for their unconditional support. Without them, this would not have been possible. AbstractThe sense of touch is especially important in task performance in unstructured environments. Many of the tasks we perform normally would be problematic, if not impossible, without sensory information of this type. Daily tasks that we take for granted, such as chewing or walking require the use of tactile information (pressure, exerted force, etc.) to be done. Thus, it is logical to think that in order to build robots capable of performing tasks in unstructured environments, it is necessary to provide them with the same information.A research area that would benefit from the inclusion of tactile information, and one that is of particular interest to robotics, is biomimetic locomotion. Improving biomimetic locomotion can impact fields such as service robotics, rehabilitation robotics, and search and rescue robots. For this reason, we will focus on biomimetic locomotion, but we will keep in other applications that can use haptic perception.Animals change their apparent leg stiffness when changing from a rigid surface to a softer one. This change is done because animals and humans maintain the same center of mass trajectory in different surfaces; thus, a change on the apparent stiffness must occur. The realization of this change requires information on the contact properties of the environment. How to extract this information is a question that still needs much research. Because of this, we have done an experimental study that compares the performance of different system identification techniques when they are applied to contact modeling. During the evaluation of the results it was found that the recursive least squares method has the best performance for haptic applications. Based on the results of this study we have selected the recursive least squares method and the spring dashpot model. With this design parameters, we devised an algorithm to be used in a robotic leg.The algorithm was implemented on a 3-degree-of-freedom (DoF) underactuated leg and then it was tested using four different terrains. The results show that the algorithm was capable of differentiating between terrains according to their stiffness and that the convergence time was under the average time a human runner is in contact with the ground. Nevertheless, the algorithm was not able to differentiate the damping coefficient in granular media. This result is to be expected because granular media...

show abstract

Ensemble-CIO: Full-body dynamic motion planning that transfers to physical humanoids

Cited by 126 publications

References 10 publications

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

Preparing for the Unknown: Learning a Universal Policy with Online System Identification

DIRTREL: Robust Trajectory Optimization with Ellipsoidal Disturbances and LQR Feedback

Parameter identification and modeling of contact properties for robotic applications

Contact Info

Product

Resources

About