Design Methodology for Deep Out-of-Distribution Detectors in Real-Time Cyber-Physical Systems

Abstract

Learning enabled components (LECs), while critical for decision making in autonomous vehicles (AVs), are likely to make incorrect decisions when presented with samples outside of their training distributions. Out-of-distribution (OOD) detectors have been proposed to detect such samples, thereby acting as a safety monitor, however, both OOD detectors and LECs require heavy utilization of embedded hardware typically found in AVs. For both components, there is a tradeoff between non-functional and functional performance, and both impact a vehicle's safety. For instance, giving an OOD detector a longer response time can increase its accuracy at the expense of the LEC. We consider an LEC with binary output like an autonomous emergency braking system (AEBS) and use risk, the combination of severity and occurrence of a failure, to model the effect of both components' design parameters on each other's functional and non-functional performance, as well as their impact on system safety. We formulate a codesign methodology that uses this risk model to find the design parameters for an OOD detector and LEC that decrease risk below that of the baseline system and demonstrate it on a vision based AEBS. Using our methodology, we achieve a 42.3% risk reduction while maintaining equivalent resource utilization.

Co-Design of Out-of-Distribution Detectors for Autonomous Emergency Braking Systems

“…), and demonstrated this on an autonomous driving dataset [7]. Although design methodologies have been proposed to optimize the execution time of such an OOD detector while respecting bounds on accuracy [8], they do not take into account the scenario where an OOD detector and an LEC share the same set of computational resources.…”

“…The default time of day and weather are used for all the gathered clips. We use the same method as [8] to augment the images with varying amounts of rain. Image augmentation is applied such that 10 rain levels ranging from 0, 0.1, 0.2, .…”