DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications

Abstract

Unsupervised Deep Learning (DL) techniques have been widely used in various security-related anomaly detection applications, owing to the great promise of being able to detect unforeseen threats and the superior performance provided by Deep Neural Networks (DNN). However, the lack of interpretability creates key barriers to the adoption of DL models in practice. Unfortunately, existing interpretation approaches are proposed for supervised learning models and/or non-security domains, which are unadaptable for unsupervised DL models and fail to satisfy special requirements in security domains.In this paper, we propose DeepAID, a general framework aiming to (1) interpret DL-based anomaly detection systems in security domains, and (2) improve the practicality of these systems based on the interpretations. We first propose a novel interpretation method for unsupervised DNNs by formulating and solving well-designed optimization problems with special constraints for security domains. Then, we provide several applications based on our Interpreter as well as a model-based extension Distiller to improve security systems by solving domain-specific problems. We apply DeepAID over three types of security-related anomaly detection systems and extensively evaluate our Interpreter with representative prior works. Experimental results show that DeepAID can provide high-quality interpretations for unsupervised DL models while meeting several special requirements in security domains. We also provide several use cases to show that DeepAID can help security operators to understand model decisions, diagnose system mistakes, give feedback to models, and reduce false positives.

SoK: Modeling Explainability in Security Monitoring for Trust, Privacy, and Interpretability

“…DeepAID [29] proposes an explanation method for unsupervised deep learning models for security applications with optimization based on specific security constraints. The idea is to find a reference normal sample x * given an anomaly x such that the difference of x * and x provides the explanation.…”

“…A few prior works on explainable security have proposed some metrics to evaluate, understand and compare explanation methods [28], [29], [86]. However, there are no universal standard metrics for comparison.…”

“…A stable method should obtain a value closer to 1. Other methods follow a similar approach with the use of Jaccard Similarity [29]. However, this metric is incomplete.…”

“…The larger the Levenshtein distance between explanations obtained for two close samples, the less continuity the method possess. The Levenshtein distance between two sequences a and b with length |a| and |b| respectively is given by lev(a, b): otherwise, (17) To compute the stability of an explanation for a sample across multiple runs, we can use Jaccard Similarity Index [29] to compare the similarity of two different results for test run i and j as:…”

“…We employ an approximation (LIME [69]) and a back-propagation (DeepAID [29]) based explanation method to produce explanations for the anomalous sequence and assist the security analyst in the decision-making. Table 1 and Table 2 visualizes the interpretation results from LIME and DeepAID respectively.…”