Derivative-Free Online Learning of Inverse Dynamics Models

Romeres, Diego; Zorzi, Mattia; Camoriano, Raffaello; Traversaro, Silvio; Chiuso, Alessandro

doi:10.1109/tcst.2019.2891222

Cited by 31 publications

(33 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We are currently working on learning a full policy using a guided policy search [14] approach, where we can optimize trajectory and policy simultaneously. Moreover, we are investigating on how to apply GP online learning techniques [41], [42], [43], [26], [44], [45] to improve the model while iteratively improving the policy. Finally, we are interested in composing the prior physics knowledge with a neural network learning framework and compare it with the proposed semiparametric GP models [46].…”

Section: Conclusion and Discussionmentioning

confidence: 99%

Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze

Romeres

Jha

Libera

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

This paper presents a problem of model learning for the purpose of learning how to navigate a ball to a goal state in a circular maze environment with two degrees of freedom. The motion of the ball in the maze environment is influenced by several non-linear effects such as dry friction and contacts, which are difficult to model physically. We propose a semiparametric model to estimate the motion dynamics of the ball based on Gaussian Process Regression equipped with basis functions obtained from physics first principles. The accuracy of this semiparametric model is shown not only in estimation but also in prediction at n-steps ahead and its compared with standard algorithms for model learning. The learned model is then used in a trajectory optimization algorithm to compute ball trajectories. We propose the system presented in the paper as a benchmark problem for reinforcement and robot learning, for its interesting and challenging dynamics and its relative ease of reproducibility.

show abstract

Section: Conclusion and Discussionmentioning

confidence: 99%

Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze

Romeres

Jha

Libera

et al. 2019

2019 International Conference on Robotics and Automation (ICRA)

Self Cite

View full text Add to dashboard Cite

show abstract

“…These approaches use any black-box function approximator as a transfer function and optimize the model parameters to fit the observed data. For example, the existing literature used Local Linear Models (Schaal et al 2002;Haruno et al 2001), Gaussian Mixture Models (Calinon et al 2010;Khansari-Zadeh and Billard 2011), Gaussian Processes (Kocijan et al 2004;Nguyen-Tuong et al 2009;Nguyen-Tuong and Peters 2010;Romeres et al 2019Romeres et al , 2016Camoriano et al 2016), Support Vector Machines (Choi et al 2007;Ferreira et al 2007), feedforward- (Jansen 1994;Lenz et al 2015;Ledezma and Haddadin 2017;Sanchez-Gonzalez et al 2018) or recurrent neural networks (Rueckert et al 2017;Ha and Schmidhuber 2018;Hafner et al 2019a,b) to learn the dynamics model. The black-box models obtain either the forward or inverse model and the learned model is only valid on the training data distribution.…”

Section: Data-driven Systemmentioning

confidence: 99%

Combining Physics and Deep Learning to learn Continuous-Time Dynamics Models

Lutter¹,

Peters²

2021

Preprint

View full text Add to dashboard Cite

Deep learning has been widely used within learning algorithms for robotics. One disadvantage of deep networks is that these networks are black-box representations. Therefore, the learned approximations ignore the existing knowledge of physics or robotics. Especially for learning dynamics models, these black-box models are not desirable as the underlying principles are well understood and the standard deep networks can learn dynamics that violate these principles. To learn dynamics models with deep networks that guarantee physically plausible dynamics, we introduce physics-inspired deep networks that combine first principles from physics with deep learning. We incorporate Lagrangian mechanics within the model learning such that all approximated models adhere to the laws of physics and conserve energy. Deep Lagrangian Networks (DeLaN) parametrize the system energy using two networks. The parameters are obtained by minimizing the squared residual of the Euler-Lagrange differential equation. Therefore, the resulting model does not require specific knowledge of the individual system, is interpretable, and can be used as a forward, inverse, and energy model. Previously these properties were only obtained when using system identification techniques that require knowledge of the kinematic structure. We apply DeLaN to learning dynamics models and apply these models to control simulated and physical rigid body systems. The results show that the proposed approach obtains dynamics models that can be applied to physical systems for real-time control. Compared to standard deep networks, the physics-inspired models learn better models and capture the underlying structure of the dynamics.

show abstract

“…Utilizing derivative-free features in the context of inverse dynamics has been proposed in [16] as a means to address the noisy nature of numerical differentiation, an inevitable step to obtain joint velocities and accelerations. Assuming we have access to the previous M joint positions, there are numerous ways to incorporate the state history as a feature.…”

Section: B Derivative-free Featuresmentioning

confidence: 99%

Cascaded Gaussian Processes for Data-efficient Robot Dynamics Learning

Rezaei-Shoshtari

Meger

Sharf

2019

2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Motivated by the recursive Newton-Euler formulation, we propose a novel cascaded Gaussian process learning framework for the inverse dynamics of robot manipulators. This approach leads to a significant dimensionality reduction which in turn results in better learning and data efficiency. We explore two formulations for the cascading: the inward and outward, both along the manipulator chain topology. The learned modeling is tested in conjunction with the classical inverse dynamics model (semi-parametric) and on its own (non-parametric) in the context of feed-forward control of the arm. Experimental results are obtained with Jaco 2 six-DOF and SARCOS seven-DOF manipulators for randomly defined sinusoidal motions of the joints in order to evaluate the performance of cascading against the standard GP learning. In addition, experiments are conducted using Jaco 2 on a task emulating a pouring maneuver. Results indicate a consistent improvement in learning speed with the inward cascaded GP model and an overall improvement in data efficiency and generalization.

show abstract

Derivative-Free Online Learning of Inverse Dynamics Models

Cited by 31 publications

References 42 publications

Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze

Semiparametrical Gaussian Processes Learning of Forward Dynamical Models for Navigating in a Circular Maze

Combining Physics and Deep Learning to learn Continuous-Time Dynamics Models

Cascaded Gaussian Processes for Data-efficient Robot Dynamics Learning

Contact Info

Product

Resources

About