Efficient Out-of-Distribution Detection Using Latent Space of β -VAE for Cyber-Physical Systems

Abstract

Deep Neural Networks are actively being used in the design of autonomous Cyber-Physical Systems (CPSs). The advantage of these models is their ability to handle high-dimensional state-space and learn compact surrogate representations of the operational state spaces. However, the problem is that the sampled observations used for training the model may never cover the entire state space of the physical environment, and as a result, the system will likely operate in conditions that do not belong to the training distribution. These conditions that do not belong to training distribution are referred to as Out-of-Distribution (OOD). Detecting OOD conditions at runtime is critical for the safety of CPS. In addition, it is also desirable to identify the context or the feature(s) that are the source of OOD to select an appropriate control action to mitigate the consequences that may arise because of the OOD condition. In this paper, we study this problem as a multi-labeled time series OOD detection problem over images, where the OOD is defined both sequentially across short time windows (change points) as well as across the training data distribution. A common approach to solving this problem is the use of multi-chained one-class classifiers. However, this approach is expensive for CPSs that have limited computational resources and require short inference times. Our contribution is an approach to design and train a single β -Variational Autoencoder detector with a partially disentangled latent space sensitive to variations in image features. We use the feature sensitive latent variables in the latent space to detect OOD images and identify the most likely feature(s) responsible for the OOD. We demonstrate our approach using an Autonomous Vehicle in the CARLA simulator and a real-world automotive dataset called nuImages.

PAC-Based Formal Verification for Out-of-Distribution Data Detection

“…Within recent years, several studies have emerged that use VAE latent encodings to reduce data dimensionality for tasks like classification [9] and anomaly detection [21]. [20] and [8] cite three clear benefits of doing so: firstly, the reduction in data dimensionality reduces the complexity of the required ML model, allowing more explainable techniques to be implemented [20]; secondly, the latent encoding allows for the quantification of high-dimensional features, increasing the robustness of classifiers applied in this space [8]; lastly, the reduction in dimensionality also reduces runtime [8].…”

“…As it is impossible to account for all possible instances and states a system may encounter during the training phase, the system's behaviour toward OOD instances cannot be anticipated accurately and can be especially undesirable in safety-critical tasks [8]. For this reason, CPSs within safety-critical domains often contain subsystems dedicated to the detection and handling of OOD data [20].…”

“…A solution to this, utilized in systems described by [8], [20] and [7], is to use variational autoencoders (VAE) to parametrize the training data distribution with a fixed number of variables. VAEs are a class of DNNs that map highdimension input data to lower-dimensional distributions that comprise latent spaces within the model.…”

“…This results in the encoding of the distribution of training data to lowerdimension multivariate distributions. Studies have made use of this property in designing OOD detection systems by attempting to equate OOD instances with outliers in the latent space [20] and constructing classifiers to define in-distribution safety constraints within this space [8].…”

“…The approach described in this study relies on the construction of constraints within the latent space that are used to define an in-distribution criteria for high-dimension data using the spatial coordinates of the encoding. Similar to [20], the constraints within this study are constructed using conformitybased classification based on a subset of known in-distribution data. Sampling the VAE latent distribution to find violations of these constraints allows for the construction of bounds on the error with which the system conducts OOD detection.…”