Safe exploration in model-based reinforcement learning using control barrier functions

Cohen, M.; Belta, Călin

doi:10.1016/j.automatica.2022.110684

Cited by 22 publications

(8 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Alternatively in some works MATLAB/ SIMULINK plat-form is also used for training or evaluating RL agents [111] 3. One crucial observation is that a huge number of work have used customized or adapted environments for training and evaluation and have not used conventional environments [74,24,84].…”

Section: Review Of Simulation/ Evaluation Benchmarksmentioning

confidence: 99%

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Banerjee,

Nguyen,

Fookes

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Review Of Simulation/ Evaluation Benchmarksmentioning

confidence: 99%

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Banerjee,

Nguyen,

Fookes

et al. 2023

Preprint

View full text Add to dashboard Cite

“…In recent years, significant progress has been made in developing safe model-based reinforcement learning (SMBRL) techniques to learn safe controllers for different classes of systems. [6][7][8][9][10][11][12][13][14][15][16] While Markov decision process (MDP) based SMBRL methods have been available for discrete time systems with finite state and action spaces, [6][7][8][9] synthesizing online controllers for systems in continuous time, under output feedback, while guaranteeing stability and safety is still a challenging problem.…”

Section: Introductionmentioning

confidence: 99%

Safe adaptive output‐feedback optimal control of a class of linear systems

Mahmud,

Abudia,

Nivison

et al. 2024

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

The objective of this research is to enable safety‐critical systems to simultaneously learn and execute optimal control policies in a safe manner to achieve complex autonomy. Learning optimal policies via trial and error, that is, traditional reinforcement learning, is difficult to implement in safety‐critical systems, particularly when task restarts are unavailable. Safe model‐based reinforcement learning techniques based on a barrier transformation have recently been developed to address this problem. However, these methods rely on full‐state feedback, limiting their usability in a real‐world environment. In this work, an output‐feedback safe model‐based reinforcement learning technique based on a novel barrier‐aware dynamic state estimator has been designed to address this issue. The developed approach facilitates simultaneous learning and execution of safe control policies for safety‐critical linear systems. Simulation results indicate that barrier transformation is an effective approach to achieve online reinforcement learning in safety‐critical systems using output feedback.

show abstract

“…For nonlinear systems, Reference 28 develops a safe exploration scheme for jointly learning the dynamics of an uncertain control system and the optimal value function/policy. The proposed approach uses Lyapunov‐like barrier functions 29 to build robust safeguarding controller that can guarantee safety when combined with an arbitrary learning‐based control policy.…”

Section: Introductionmentioning

confidence: 99%

Off‐policy model‐based end‐to‐end safe reinforcement learning

Kanso,

Jha,

Theilliol

2023

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Safety and stability considerations play a crucial role in the development of learning based strategies for control design of systems that require high levels of safety. Safe reinforcement learning (RL) based approaches traditionally seek learning of the control laws that are optimal with respect to system performance whilst ensuring system stability and safety. In this article, an off‐policy safe RL based approach is proposed for nonlinear systems affine in control in continuous time. In this novel work, safety and stability are guaranteed during initialization and exploration phases by adjusting the control input with the solution of a quadratic programming problem combining both input to state stable‐control Lyapunov function and robust control barrier function (R‐CBF) conditions. Moreover, the safety of the learned policy is assured by augmenting the cost function with a CBF to maintain safety and optimize performance simultaneously. Novel mathematically rigorous proofs are provided to establish the stability and safety guarantees, offering a sound theoretical foundation for the approach. To demonstrate the effectiveness of the algorithm, two examples are presented: engine surge and stall dynamics, and an unstable nonlinear system.

show abstract

Safe exploration in model-based reinforcement learning using control barrier functions

Cited by 22 publications

References 68 publications

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

A Survey on Physics Informed Reinforcement Learning: Review and Open Problems

Safe adaptive output‐feedback optimal control of a class of linear systems

Off‐policy model‐based end‐to‐end safe reinforcement learning

Contact Info

Product

Resources

About