Anomaly Detection Through Latent Space Restoration Using Vector Quantized Variational Autoencoders

Abstract

We propose an out-of-distribution detection method that combines density and restoration-based approaches using Vector-Quantized Variational Auto-Encoders (VQ-VAEs). The VQ-VAE model learns to encode images in a categorical latent space. The prior distribution of latent codes is then modelled using an Auto-Regressive (AR) model. We found that the prior probability estimated by the AR model can be useful for unsupervised anomaly detection and enables the estimation of both sample and pixel-wise anomaly scores. The sample-wise score is defined as the negative loglikelihood of the latent variables above a threshold selecting highly unlikely codes. Additionally, out-of-distribution images are restored into in-distribution images by replacing unlikely latent codes with samples from the prior model and decoding to pixel space. The average L1 distance between generated restorations and original image is used as pixelwise anomaly score. We tested our approach on the MOOD challenge datasets, and report higher accuracies compared to a standard reconstruction-based approach with VAEs.

Diffusion Models for Medical Anomaly Detection

“…Any deviations from the learned distribution then lead to a high anomaly score. This idea has been applied for unsupervised anomaly detection in medical images [29,6,14], where the difference between the healthy reconstruction and the anomalous input image highlight pixels that are perceived as anomalous. Other approaches focus on Generative Adversarial Networks (GANs) [9] for image-to-image translation [24,5,27].…”

nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methods

“…The assumption is that a model trained on normal data will not be able to reproduce anomalous regions, leading them to larger deviations. This method lends itself most easily to autoencoder-based architectures [3,15], but has also been applied by using generative adversarial networks [11,20]. However, reconstruction loss often fails as models are not always able to reconstruct healthy regions in potentially unhealthy samples [2], and anomalies with extreme textures but a normal intensity distribution are difficult to identify [16].…”

“…Many publications in the field of medical anomaly detection limit their experiments to datasets with a narrow range of abnormalities [3,15,20,24], raising questions regarding their ability to generalise to anomalies seen in other medical applications or modalities. Recently there has been a trend of training end-toend anomaly detection models using synthetic anomalies to alter healthy data.…”

Dual-Distribution Discrepancy for Anomaly Detection in Chest X-Rays

“…They train the reconstruction networks like variants of autoencoder (AE) to minimize the reconstruction error on normal images, while unseen abnormal images are assumed not able to be reconstructed, and in turn yield larger reconstruction errors. To avoid the reconstruction of anomaly and reduce miss detection, some methods [5,14] utilize Variational AE (VAE) [8] to approximate the normative distribution and perform image restoration iteratively to ensure the output is anomaly-free, yielding higher difference with the abnormal input. However, the iterative restoration process is computationally complex and time-consuming.…”