Temporal-difference learning for online reachability analysis

Abstract

Hamilton-Jacobi-Isaacs (HJI) reachability analysis has been employed to guarantee safety in a number of applications including robotics, air traffic control, and control of HVAC systems. The current standard for these methods can result in overly-conservative controllers that degrade system performance with respect to other objectives. There has been interest in incorporating online machine learning techniques to reduce the conservativeness of the controller. However, recent efforts have resulted in methods that are computationally inefficient and scale poorly with the dimension of the state space. We propose a novel online reachability update algorithm based on Temporal-Difference (TD) learning that is computationally more efficient. Our algorithm is demonstrated on a simulation of a quadrotor learning to track a trajectory in a confined space. Our method outperforms standard reachability-based controllers when it comes to other (non-safety) objectives.

Scalable Learning of Safety Guarantees for Autonomous Systems using Hamilton-Jacobi Reachability

“…Contact info: sherbert@ucsd.edu, jason.choi@berkeley.edu, sqs@cs.cmu.edu, mtgibson@berkeley.edu, koushils@berkeley.edu, tomlin@berkeley.edu model or constraints no longer hold. Therefore, the system must update its assumptions and corresponding safe methods in an online fashion as the system learns more about the environment [6][7][8][9][10][11].…”