Enforcing Hard State-Dependent Action Bounds on Deep Reinforcement Learning Policies

Abstract

Imposing hard constraints on deep reinforcement learning policies trained with model-free methods is a challenging task. In this paper we specifically focus on constraining the policy's actions, by imposing state-dependent action bounds. Such bounds allow the designer to incorporate prior domain knowledge into the model-free learning framework and can be used to improve the stability or safety of the learned policies. The approach is applied to two benchmark environments and a more complicated autonomous driving problem. When correctly applied, the state-dependent action bounds can provide strong safety guarantees, as well as improve the convergence speed.