Nonlinear Control Synthesis Under Double Constraints

Daryin, Alexander N.; Kurzhanski, Alexander B.

doi:10.3182/20050703-6-cz-1902.00441

Cited by 1 publication

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Inspired by the ideas from [18] we adapt it to the RL case, we propose to reduce the problem to an MDP. In particular, we remove the constraint by reshaping the cost and using state augmentation.…”

Section: Main Idea In the Deterministic Casementioning

confidence: 99%

See 1 more Smart Citation

SAUTE RL: Almost Surely Safe Reinforcement Learning Using State Augmentation

Sootla¹,

Cowen-Rivers²,

Jafferjee³

et al. 2022

Preprint

View full text Add to dashboard Cite

Satisfying safety constraints almost surely (or with probability one) can be critical for deployment of Reinforcement Learning (RL) in reallife applications. For example, plane landing and take-off should ideally occur with probability one. We address the problem by introducing Safety Augmented (Saute) Markov Decision Processes (MDPs), where the safety constraints are eliminated by augmenting them into the state-space and reshaping the objective. We show that Saute MDP satisfies the Bellman equation and moves us closer to solving Safe RL with constraints satisfied almost surely. We argue that Saute MDP allows to view Safe RL problem from a different perspective enabling new features. For instance, our approach has a plug-and-play nature, i.e., any RL algorithm can be "sauteed". Additionally, state augmentation allows for policy generalization across safety constraints. We finally show that Saute RL algorithms can outperform their state-of-the-art counterparts when constraint satisfaction is of high importance. * Honorary position Preprint. Under review.

show abstract

Section: Main Idea In the Deterministic Casementioning

confidence: 99%

“…This idea can be traced back to classical control methods of augmenting the safety constraints into the state-space cf. [18], however, we adapt it to stochastic problems and RL applications. First, we reshape the objective to take into account the remaining safety budget by assigning infinite values if the budget was spent.…”

Section: Introductionmentioning

confidence: 99%