Full-Spectrum Out-of-Distribution Detection

Abstract

Existing out-of-distribution (OOD) detection literature clearly defines semantic shift as a sign of OOD but does not have a consensus over covariate shift. Samples experiencing covariate shift but not semantic shift are either excluded from the test set or treated as OOD, which contradicts the primary goal in machine learning-being able to generalize beyond the training distribution. In this paper, we take into account both shift types and introduce full-spectrum OOD (FS-OOD) detection, a more realistic problem setting that considers both detecting semantic shift and being tolerant to covariate shift; and designs three benchmarks. These new benchmarks have a more finegrained categorization of distributions (i.e., training ID, covariate-shifted ID, near-OOD, and far-OOD) for the purpose of more comprehensively evaluating the pros and cons of algorithms. To address the FS-OOD detection problem, we propose SEM, a simple feature-based semantics score function. SEM is mainly composed of two probability measures: one is based on high-level features containing both semantic and non-semantic information, while the other is based on low-level feature statistics only capturing non-semantic image styles. With a simple combination, the non-semantic part is cancelled out, which leaves only semantic information in SEM that can better handle FS-OOD detection. Extensive experiments on the three new benchmarks show

Out-of-Distribution (OOD) Detection and Generalization Improved by Augmenting Adversarial Mixup Samples

“…This paper proposes a distance-aware adversarial mixup (AM) OOD detector to classify semantic OOD samples while being robust to negligible covariate shift. Full spectrum OOD detection highlights the effects of covariate-shifted in-distribution and shows that most existing OOD detectors are susceptible to covariate shift rather than semantic shift [3,29]. This paper proposes a simple and novel OOD sample augmentation technique to increase the generalization capability of OOD detectors and analyzes how OOD detection and OOD generalization can better enable each other, in terms of both algorithmic design and comprehensive performance evaluation [3].…”