An Imputation-based Augmented Anomaly Detection from Large Traces of Operating System Events

Ezeme, Okwudili M.; Azim, Akramul; Mahmoud, Qusay H.

doi:10.1145/3148055.3148076

Cited by 9 publications

(17 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Because our work centers on the anomaly detection approach, we highlight mostly the practices in this area. In [13], [14], the authors used kernel events to build an offline anomaly detection model using some vector space model concepts and agglomerative clustering technique. Imputation techniques were also used to increase the scope of their model and reduce the incidents of false positives.…”

Section: Related Workmentioning

confidence: 99%

“…Imputation techniques were also used to increase the scope of their model and reduce the incidents of false positives. However, the anomaly frameworks of [13], [14] lack temporal modeling; hence, they do not capture the nature of the system process behavior that emits system calls in discrete, sequential mode. Reference [15] used deep LSTM models constructed from system logs to create anomaly detection models for detecting anomalies in logs from virtual machines.…”

Section: Related Workmentioning

confidence: 99%

“…Authors of [19] have an anomaly model built using system call frequency distribution and clustering techniques to detect when processes deviate from their standard profiles. Similar to the anomaly frameworks of [13], [14], the model of [19] has no temporal modeling of the events and can only be used for post-mortem analysis. Furthermore, the authors of [20] used the statistical metric of entropy to implement an anomaly detection model for network logs.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Design and Development of AD-CGAN: Conditional Generative Adversarial Networks for Anomaly Detection

2020

Self Cite

View full text Add to dashboard Cite

Whether in the realm of software or hardware, datasets representing the state of systems are mostly imbalanced. This imbalance is because these systems' reliability requirements make the occurrence of an anomaly a rare phenomenon. Hence, most datasets on anomaly detection have a relatively small percentage that captures the anomaly. Recently, generative adversarial networks (GAN) have shown promising results in image generation tasks. Therefore, in this research work, we build on conditional GANs (CGAN) to generate plausible distributions of a given profile to solve the challenge of data imbalance in anomaly detection tasks and present a novel framework for anomaly detection. Firstly, we learn the pattern of the minority class data samples using a single class CGAN. Secondly, we use the knowledge base of the single class CGAN to generate samples that augment the minority class samples so that a binary class CGAN can train on the typical and malicious profiles with a balanced dataset. This approach inherently eliminates the bias imposed on algorithms from the dataset and results in a robust framework with improved generalization. Thirdly, the binary class CGAN generates a knowledge base that we use to construct the cluster-based anomaly detector. During testing, we do not use the single class CGAN, thereby providing us with a lean and efficient algorithm for anomaly detection that can do anomaly detection on semisupervised and non-parametric multivariate data. We test the framework on logs and image-based anomaly detection datasets with class imbalance. We compare the performance of AD-CGAN with GAN-derived and non-GAN-derived state of the art algorithms on benchmark datasets. AD-CGAN outperforms most of the algorithms in the standard metrics of Precision, Recall, and F-1 Score. Where AD-CGAN does not perform better in the parameters used, it has the advantage of being lightweight. Therefore, it can be deployed for both online and offline anomaly detection tasks since it does not use an input sample inversion strategy.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Design and Development of AD-CGAN: Conditional Generative Adversarial Networks for Anomaly Detection

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…The obvious limitation of this method is that zero-day vulnerabilities cannot be detected as it only searches for known signatures. On the other hand, Ezeme et al [9], Xu et al [10], Du et al [16], Yu et al [17] use the second approach which involves the construction of a model to target both seen and unseen anomalies. The core principle of these approaches consists of the extraction of features from the operational profile of the system to construct models which can differentiate normal and anomalous behavior.…”

Section: Related Workmentioning

confidence: 99%

“…Also, anomaly detection models based on entropy like [8] does not fit this kind of analysis because it relies on volume to detect an anomaly rather than the temporal information conveyed by the tuples. Also, the variants of the vector space models used in [9]- [11] cannot be used for online anomaly detection because it performs classification based on a large buffer of the traces.…”

Section: Introductionmentioning

confidence: 99%

DReAM: Deep Recursive Attentive Model for Anomaly Detection in Kernel Events

2019

Self Cite

View full text Add to dashboard Cite

System logs and traces contain information that reflects the state of the system and serves as a rich source of knowledge for system monitoring from the application to the kernel layer. Moreover, logging of traces as a tool for monitoring the operation of a cyber-physical system is recommended by most safety standard organizations. However, because the data can be overwhelmingly huge within a short space of time, the use of models that do not rely only on known signatures for online anomaly detection becomes difficult to use due to the challenge of processing such an enormous amount of data at runtime. Hence, most practitioners resort to the use of signature-based tools. In this paper, we introduce an anomaly detection model that uses intra-trace and inter-trace context vectors with long short-term memory networks to overcome the challenge of online anomaly detection in cyber-physical systems. We test the performance of the model with publicly available datasets that reflect the internal and external control flow of an embedded application and our model demonstrates both the effectiveness and robustness in detecting an anomalous sequence in a system call stream.INDEX TERMS Anomaly detection, kernel events, embedded system, machine learning.

show abstract