A robust stability approach to robot reinforcement learning based on a parameterization of stabilizing controllers

Friedrich, Stefan; Buss, Martin

doi:10.1109/icra.2017.7989382

Cited by 12 publications

(12 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This way, it is always possible to achieve < 1 in (9). In the authors' experience, 11 this approach constitutes a viable trade-off to build a parameterization-based robustly stable learning control system that noticeably exploits domain knowledge without a detailed dynamical model of the robot manipulator at hand.…”

Section: Based On Crude Approximationmentioning

confidence: 99%

“…To quantify the uncertainty in the closed loop bŷS, an expression for the corresponding dual-Youla parameter is derived as well. (25), (28) controlled by (37) is given as where A S,11 = B 13 D 0 C 31 + A 11 , Proof. The derivation is analogous to the full case of Theorem 2, replacing the coprime factors with those for a static controller.…”

Section: Special Case: Static Nominal Controlmentioning

confidence: 99%

“…The derivation is analogous to the full case of Theorem 2, replacing the coprime factors with those for a static controller. 11 Alternatively, the given realization of S can be obtained from (33) by appropriately removing obsolete rows and columns for undefined A K , B K , C K , and taking D K = D 0 .…”

Section: Special Case: Static Nominal Controlmentioning

confidence: 99%

“…8 The primary parameterization was already applied to the robust linear design of robot controllers around 30 years ago, 9 known in robotics as the stable factorization approach. 9,10 Our interest in extending this approach is due to the beneficial stability and robustness properties obtained by a Q-parameterization: Our recent work 11 suggests that such a parameterization is very helpful to leverage machine learning also in a feedback configuration, that is, in closed loop and on hardware. Inherently, the question is raised of how much feedback controller modification is admissible, given the effect of neglected uncertainties in the closed-loop system.…”

mentioning

confidence: 99%

“…,11 are the insertion of the coprime factors U 0 andM 0 from (B4) and algebraic simplification, subsequent application of a state similarity transformation such that = Tx with of four uncontrollable and one unobservable modes. A realization is finally obtained as By analogous steps, for T Δ,12 and T Δ,21 , we have where one uncontrollable and one unobservable mode were removed, respectively.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Parameterizing robust manipulator controllers under approximate inverse dynamics: A double‐Youla approach

Friedrich

Buss

2019

Intl J Robust & Nonlinear

Self Cite

View full text Add to dashboard Cite

We consider the goal of ensuring robust stability when a given manipulator feedback control law is modified online, for example, to safely improve the performance by a learning module. To this end, the factorization approach is applied to both the plant and controller models to characterize robustly stabilizing controllers for rigid-body manipulators under approximate inverse dynamics control. Outer-loop controllers to stabilize the nonlinear uncertain loop that results from approximate inverse dynamics are often derived by lumping uncertainty in a single term and subsequent analysis of the error system. Here, by contrast, the well-known norm bounds of these uncertain dynamics are first recast into a generalized plant configuration that preserves the characteristic uncertainty structure. Then, the overall loop uncertainty is expressed with respect to the nominal outer-loop feedback controller by means of an uncertain dual-Youla operator. Therefore, using the dual-Youla parameterization, we provide a novel way to rigorously quantify permissible perturbations of robot manipulator feedforward/feedback controllers. The method proposed in this paper does not constitute another robust control law for rigid-body manipulators, but rather a characterization of a set of robustly stabilizing controllers. The resulting double-Youla parameterization for the control of robot manipulators is amenable to numerous advanced design methods. The result is thoroughly discussed by a planar elbow manipulator and exemplified with a six-degree-of-freedom robot scenario with varying payload. KEYWORDS approximate inverse dynamics, dual-Youla parameterization, robust robot manipulator control, robust/adaptive control, uncertainty quantification INTRODUCTIONOperating conditions of robot manipulators change over time, impairing the control performance, for example, by wear, varying load, etc. To adapt the controller to such situations, a variety of data-driven methods have been proposed in the literature, including adaptive and learning control. [1][2][3][4] In practice, however, very simple feedback controllers and model-based architectures are prevalent.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

show abstract

Section: Based On Crude Approximationmentioning

confidence: 99%

Section: Special Case: Static Nominal Controlmentioning

confidence: 99%

Section: Special Case: Static Nominal Controlmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Parameterizing robust manipulator controllers under approximate inverse dynamics: A double‐Youla approach

Friedrich

Buss

2019

Intl J Robust & Nonlinear

Self Cite

View full text Add to dashboard Cite

show abstract

Learning-Based Parameter Optimization for a Class of Orbital Tracking Control Laws

Bianchini,

Garulli,

Giannitrapani

et al. 2024

J Astronaut Sci

View full text Add to dashboard Cite

This paper presents a learning algorithm for tuning the parameters of a family of stabilizing nonlinear controllers for orbital tracking, in order to minimize a cost function which combines convergence time and fuel consumption. The main feature of the proposed approach is that it achieves performance optimization while guaranteeing closed-loop stability of the resulting controller. This property is exploited also to restrict the class of admissible controllers and hence to expedite the training process. The learning algorithm is tested on three case studies: two different orbital transfers and a rendezvous mission. Numerical simulations show that the learned control parameters lead to a significant improvement of the considered performance measure.

show abstract

Reinforcement learning for control: Performance, stability, and deep approximators

Buşoniu

Bruin

Tolić

et al. 2018

Annual Reviews in Control

343

141

View full text Add to dashboard Cite

Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL; and we close the survey with an outlook that -among other things -points out some avenues for bridging the gap between control and artificial-intelligence RL techniques.

show abstract

A robust stability approach to robot reinforcement learning based on a parameterization of stabilizing controllers

Cited by 12 publications

References 15 publications

Parameterizing robust manipulator controllers under approximate inverse dynamics: A double‐Youla approach

Parameterizing robust manipulator controllers under approximate inverse dynamics: A double‐Youla approach

Learning-Based Parameter Optimization for a Class of Orbital Tracking Control Laws

Reinforcement learning for control: Performance, stability, and deep approximators

Contact Info

Product

Resources

About