Semantic enhanced for out-of-distribution detection

Abstract

While improving the performance on the out-of-distribution (OOD) benchmark dataset, the existing approach misses a portion of the valid discriminative information such that it reduces the performance on the same manifold OOD (SMOOD) data. The key to addressing this problem is to prompt the model to learn effective and comprehensive in-distribution (ID) semantic features. In this paper, two strategies are proposed to improve the generalization ability of the model to OOD data. Firstly, the original samples are replaced by features extracted from multiple “semantic perspectives” to obtain a comprehensive semantics of the samples; Second, the mean and variance of the batch samples are perturbed in the inference stage to improve the sensitivity of the model to the OOD data. The method we propose does not employ OOD samples, uses no pre-trained models in training, and does not require pre-processing of samples during inference. Experimental results show that our method enhances the semantic representation of ID data, surpasses state-of-the-art detection results on the OOD benchmark dataset, and significantly improves the performance of the model in detecting the SMOOD data.

Learning by Erasing: Conditional Entropy Based Transferable Out-of-Distribution Detection

“…Zhang et al (Zhang et al 2020) find that the representations of inputs in DGMs can be approximated by fitted Gaussian and the distance between the distribution of representations of inputs and prior of the ID dataset can be utilized to detect OOD samples. Jiang et al (Jiang, Sun, and Yu 2022) propose to compare the training and test samples in the latent space of a flow model. However, these methods require retraining when encountering new ID datasets, which is computationally expensive and time-consuming.…”

“…Since it is time-consuming and laborious to obtain labeled data in real scenarios, as an alternative, Deep Generative Models (DGMs) have been used to capture the sample distribution of In-Distribution (ID) datasets (Serrà et al 2020). However, most DGMs-based methods focus on elaborating architectures (Ren et al 2019;Serrà et al 2020), designing loss functions (Xiao, Yan, and Amit 2020) or statistical models (Zhang et al 2020;Jiang, Sun, and Yu 2022), targeting the specific feature representation or data distribution of ID samples (Sun et al 2023), i.e., need retraining to adapt to the normal pattern of the new ID datasets. This motivates the following unexplored question: How can we make OOD detection transferable across new ID datasets?…”

Rule-based Out-Of-Distribution Detection

“…The method collects a bunch of operational data before processing and classifying them as in or out of distribution. For this reason, it falls in the category of groupwise methods [22]. Differently from pointwise, groupwise confirms a type of situation (in or out), without relying on a single point that could be a spike in a steady trend.…”

A deep generative prior for high-resolution isotropic MR head slices

“…Deep generative models have received considerable attention over the past several years, with applications in data augmentation, 1,2 out-of-distribution detection, [3][4][5] and as a prior for compressed sensing. [6][7][8][9] To support this, a highresolution (HR) generative prior for medical imaging is needed and, accordingly, many papers have explored the training and improvement of such models.…”