Optimal control of a two‐wheeled self‐balancing robot by reinforcement learning

Guo, Linyuan; Rizvi, Syed Ali Asad; Lin, Zongli

doi:10.1002/rnc.5058

Cited by 26 publications

(11 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Next, combining inequalities (15) and (16) and using (10), it may be deduced that the estimation error x(t) and the adaptation error ρ(t) converge to a compact set whose radius may be reduced by choosing a sufficiently large value of the design parameter θ.…”

Section: Adaptive Observer For States and Terrain Inclinationmentioning

confidence: 99%

“…Furthermore, common networked control strategies have been implemented in [15] for stabilizing a two-wheeled inverted pendulum robot over a wireless channel despite time-varying delays and paket loss. Also, a new feedback reinforcement learning method was proposed in [16] to solve the LQR control problem for the two-wheeled self-balancing robot. e suggested method scheme was completely online and did not require any knowledge of the system parameters.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Adaptive Observer-Based Sliding Mode Control for a Two-Wheeled Self-Balancing Robot under Terrain Inclination and Disturbances

Jmel

Dimassi

Hadj-Said

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

This paper investigates an output feedback sliding mode control scheme for a two-wheeled self-balancing robot under terrain inclination and disturbances. First of all, an adaptive high-gain observer is designed for the robot to estimate, simultaneously, the unmeasured states and the unknown terrain inclination angle which appears nonlinearly in the dynamics of the wheeled robot, using the only measured linear and angular positions. Then, the estimated states and the reconstructed unknown inclination angle are used by an appropriate continuously implemented sliding mode controller whose the design is based on the boundary layer approximation approach to reduce the chattering phenomenon. The objective of the proposed robust controller is to ensure the tracking control of the two-wheeled robot despite the unknown terrain inclination and the presence of friction disturbances. The stability of the adaptive observer-based output feedback system is established through a Lyapunov analysis, and it is inspired from sliding modes theory. Numerical simulations results highlight the effectiveness of the proposed tracking control scheme applied on two-wheeled self-balancing robot subject to terrain inclination even in the presence of unavailable disturbances.

show abstract

Section: Adaptive Observer For States and Terrain Inclinationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Adaptive Observer-Based Sliding Mode Control for a Two-Wheeled Self-Balancing Robot under Terrain Inclination and Disturbances

Jmel

Dimassi

Hadj-Said

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…Recently, reinforcement learning (RL) and approximate/adaptive dynamic programming (ADP) algorithms are widely applied to solve optimal control problems. [14][15][16][17][18][19] Moreover, for MJSs, the optimal controller can also be obtained through online learning by using RL and ADP algorithm. In He et al, 20 an online adaptive optimal control scheme is developed by a novel policy iteration algorithm.…”

Section: Introductionmentioning

confidence: 99%

Model-free adaptive optimal control policy for Markov jump systems: A value iterations algorithm

Zhou

Wen

Swain

et al. 2022

Proceedings of the Institution of Mechanical Engineers, Part I:

View full text Add to dashboard Cite

This article develops a model-free adaptive optimal control policy for discrete-time Markov jump systems. First, a two-player zero-sum game is formulated to obtain an optimal control policy that minimizes a cost function against the worst-case disturbance. Second, an action and mode-dependent value function is set up for zero-sum game to search such a policy with convergence guarantee rather than solving an optimization problem satisfying coupled algebraic Riccati equations. To be specific, motivated by the Bellman optimal principle, we develop an online value iterations algorithm to solve the zero-sum game, which is learning while controlling without any initial stabilizing policy. By this algorithm, we can achieve disturbance attenuation for Markov jump systems without knowledge of the system matrices. The adaptivity to slowly changing uncertainties can also be achieved due to the model-free feature and policy convergence. Finally, the effectiveness and practical potential of the algorithm are demonstrated by considering two numerical examples and a solar boiler system.

show abstract

“…The disadvantage of this approach is the need for an explicit knowledge of the system model; therefore, model-free methods working based on the concept of reinforcement learning (RL) have been proposed in the literature. This method has been effectively used to solve the regulation problem for the nonlinear systems and learn the optimal control solution in real-time while cutting the need for the complete knowledge of the system dynamics [5]- [12].…”

Section: Introductionmentioning

confidence: 99%

Regulation With Guaranteed Convergence Rate for Continuous-Time Systems With Completely Unknown Dynamics in the Presence of Disturbance

et al. 2022

View full text Add to dashboard Cite

This paper presents the design of a novel H ∞ -based control framework for state regulation of continuous-time linear systems with completely unknown dynamics. The proposed method solves the regulation problem with the desired convergence rate and simultaneously seeks to attenuate the adverse effect of disturbance on the system. The H ∞ regulation problem assumes a cost function that considers regulation with a guaranteed rate of convergence as well as disturbance attenuation. The problem is then turned into a two-player zero-sum game optimization problem that can be solved by solving the associated algebraic Riccati equation (ARE), which provides a model-based solution. To solve this problem in a modelfree way, a novel integral reinforcement learning (IRL) algorithm is designed to learn the solution online without requiring any prior knowledge of the system dynamics. It is shown that the model-free method (i.e., IRL-based method) provides the same solution as the model-based method (i.e., ARE). The effectiveness of the proposed method is ascertained through simulation examples; it is shown that the proposed method effectively addresses the problem for both stable and unstable systems.

show abstract

Optimal control of a two‐wheeled self‐balancing robot by reinforcement learning

Cited by 26 publications

References 20 publications

Adaptive Observer-Based Sliding Mode Control for a Two-Wheeled Self-Balancing Robot under Terrain Inclination and Disturbances

Adaptive Observer-Based Sliding Mode Control for a Two-Wheeled Self-Balancing Robot under Terrain Inclination and Disturbances

Model-free adaptive optimal control policy for Markov jump systems: A value iterations algorithm

Regulation With Guaranteed Convergence Rate for Continuous-Time Systems With Completely Unknown Dynamics in the Presence of Disturbance

Contact Info

Product

Resources

About