Perfect Density Models Cannot Guarantee Anomaly Detection

Abstract

Thanks to the tractability of their likelihood, several deep generative models show promise for seemingly straightforward but important applications like anomaly detection, uncertainty estimation, and active learning. However, the likelihood values empirically attributed to anomalies conflict with the expectations these proposed applications suggest. In this paper, we take a closer look at the behavior of distribution densities through the lens of reparametrization and show that these quantities carry less meaningful information than previously thought, beyond estimation issues or the curse of dimensionality. We conclude that the use of these likelihoods for anomaly detection relies on strong and implicit hypotheses, and highlight the necessity of explicitly formulating these assumptions for reliable anomaly detection.

Water distribution pipe lifespans: Predicting when to repair the pipes in municipal water distribution networks using machine learning techniques

“…The use of anomaly detection algorithms, as opposed to probability laws, for predicting lifespan is justified by several factors [33][34][35]: Anomaly detection algorithms excel at identifying unusual patterns in data, which can be indicative of potential system failures or abnormalities. Unlike probability laws, these algorithms do not rely on prior knowledge or accumulated data,…”

Data Informed Decision Bias

“…One of the continuing unsolved issues in deep generative modelling has been the out-of-distribution problem [171]. In particular, the model assigns high probability ("overconfidently wrong predictions") to datapoints that were unseen during training-which is both problematic and unintuitive [208].…”