Anomalous Example Detection in Deep Learning: A Survey

Abstract

Deep Learning (DL) is vulnerable to out-of-distribution and adversarial examples resulting in incorrect outputs. To make DL more robust, several posthoc (or runtime) anomaly detection techniques to detect (and discard) these anomalous samples have been proposed in the recent past. This survey tries to provide a structured and comprehensive overview of the research on anomaly detection for DL based applications. We provide a taxonomy for existing techniques based on their underlying assumptions and adopted approaches. We discuss various techniques in each of the categories and provide the relative strengths and weaknesses of the approaches. Our goal in this survey is to provide an easier yet better understanding of the techniques belonging to different categories in which research has been done on this topic. Finally, we highlight the unsolved research challenges while applying anomaly detection techniques in DL systems and present some high-impact future research directions.

Application of Improved Asynchronous Advantage Actor Critic Reinforcement Learning Model on Anomaly Detection

“…Comprehensive surveys on network anomaly detection including algorithms, experiments and analyses were done in [10][11][12][13]. Deep Learning for anomaly detection were surveyed in [14,15]. Researches were on sequential anomaly detection using RL [16,17].…”

Minority Resampling Boosted Unsupervised Learning With Hyperdimensional Computing for Threat Detection at the Edge of Internet of Things

“…Machine learning has proven to be far more effective than knowledge-based and statistical techniques for anomaly detection [37]. Existing literature reports all three types of machine learning for anomaly detection, supervised, unsupervised, and semi-supervised [38]. Supervised learning for anomaly detection is affected by the need for pre-labelled data of normal and anomalous behaviors.…”

“…Knowledge-based and statistical approaches are affected by the limitations of capturing, profiling and updating IoT Edge configurations at operation level in a dynamic computing environment, and the exposure of system vulnerabilities for behaviour profiling, whereas machine learning is able to address these limitations by managing the adaptive disposition and dynamic behavior of IoT Edge operations with high detection rates, low false positives and pragmatic computation and communication costs [18], [19]. More specifically, unsupervised machine learning methods are technically suited for the detection of behaviour-based cyber threat and attacks on IoT Edge as it can learn from unlabeled data [20]. In settings where machine learning is based on imbalanced datasets, the weakness of general learning algorithms contributes to the difficulties of classifying the anomalies as the algorithms generally bias towards the majority class samples.…”

Embedded out-of-distribution detection on an autonomous robot platform

“…One emerging approach is to proactively detect whether distribution shift occurs in real-time input data and devise a fail-safe mechanism to prevent CPS, for example, an ML-enabled autonomous driving module, from acting on unreliable ML outputs. Along this line, many out-of-distribution (OOD) detection methods [2,7,20] have been proposed in the literature modeling from a supervised or unsupervised learning principle.…”