EXAD: A System for Explainable Anomaly Detection on Big Data Traces

Song, Fei; Diao, Yanlei; Read, Jesse; Stiegler, Arnaud; Bifet, Albert

doi:10.1109/icdmw.2018.00204

Cited by 23 publications

(19 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [52] the authors perform anomaly detection using a LSTM neural network. They then approximate the neural network by a decision tree in order to retrieve the explanations.…”

Section: Anomaly Explanation By Feature Valuesmentioning

confidence: 99%

Anomaly explanation: A review

Tchaghe

Smits

Pivert

2022

Data & Knowledge Engineering

View full text Add to dashboard Cite

Anomaly detection has been studied intensively by the data mining community for several years. As a result, many methods to detect anomalies have emerged, and others are still under development. But during the recent years, anomaly detection, just like a lot of machine learning tasks, is facing a wall. This wall, erected by the lack of trust of the final users, has slowed down the usage of these algorithms in the real-world situations for which they are designed. Having the best empirical accuracy is not enough anymore; there is a need for algorithms to explain their outputs to the users in order to increase their trust. Consequently, a new expression has emerged recently: eXplainable Artificial Intelligence (XAI). This expression, which gathers all the methods that provide explanations to the output of algorithms has gained popularity, especially with the outbreak of deep learning. A lot of work has been devoted to anomaly detection in the literature, but not as much to anomaly explanation. There is so much work on anomaly detection that several reviews can be found on the topic. In contrast, we were not able to find a survey on anomaly explanation in particular, while there are a lot of surveys on XAI in general or on XAI for neural networks for example. With this paper, we want to provide a comprehensive review of the anomaly explanation field. After a brief recall of some important anomaly detection algorithms, the anomaly explanation methods that we discovered in the literature will be classified according to a taxonomy that we define. This taxonomy stems from an analysis of what is really important when trying to explain anomalies.

show abstract

“…In [52] the authors perform anomaly detection using a LSTM neural network. They then approximate the neural network by a decision tree in order to retrieve the explanations.…”

Section: Anomaly Explanation By Feature Valuesmentioning

confidence: 99%

Anomaly explanation: A review

Tchaghe

Smits

Pivert

2022

Data & Knowledge Engineering

View full text Add to dashboard Cite

show abstract

“…Some techniques limit the search space by using a heuristic approach; however, the approach does not guarantee to find the optimum subspace, which is the subspace where the outlier is the most abnormal [47,50]. Instead of searching the subspace, some techniques [24,68] depend on the local neighborhood of each outlier to extract its outlying attributes, and some other techniques [89,90] use interpretable models to measure the contribution of each feature/attribute to the abnormality of the object.…”

Section: Challenges In Generating Outlying Attributes Of An Individua...mentioning

confidence: 99%

A survey on outlier explanations

et al. 2022

View full text Add to dashboard Cite

While many techniques for outlier detection have been proposed in the literature, the interpretation of detected outliers is often left to users. As a result, it is difficult for users to promptly take appropriate actions concerning the detected outliers. To lessen this difficulty, when outliers are identified, they should be presented together with their explanations. There are survey papers on outlier detection, but none exists for outlier explanations. To fill this gap, in this paper, we present a survey on outlier explanations in which meaningful knowledge is mined from anomalous data to explain them. We define different types of outlier explanations and discuss the challenges in generating each type. We review the existing outlier explanation techniques and discuss how they address the challenges. We also discuss the applications of outlier explanations and review the existing methods used to evaluate outlier explanations. Furthermore, we discuss possible future research directions.

show abstract

“…Methods such as Shapley values [99] and region partition trees [100] have been successfully applied to explain detected anomalies. More systemic approaches have been developed too, such as Exathlon [101], EXAD [102], and others [103]. While EXAD focuses on explanation discovery for each anomaly, Exathlon crafts the explanations, providing two pieces of information to the user: why the data point was identified as an anomaly and the root causes of such anomaly.…”

Section: Explainable Artificial Intelligencementioning

confidence: 99%

Explaining Bad Forecasts in Global Time Series Models

et al. 2021

View full text Add to dashboard Cite

While increasing empirical evidence suggests that global time series forecasting models can achieve better forecasting performance than local ones, there is a research void regarding when and why the global models fail to provide a good forecast. This paper uses anomaly detection algorithms and explainable artificial intelligence (XAI) to answer when and why a forecast should not be trusted. To address this issue, a dashboard was built to inform the user regarding (i) the relevance of the features for that particular forecast, (ii) which training samples most likely influenced the forecast outcome, (iii) why the forecast is considered an outlier, and (iv) provide a range of counterfactual examples to understand value changes, in the feature vector or the predicted value, can lead to a different outcome. Moreover, a modular architecture and a methodology were developed to iteratively remove noisy data instances from the train set, to enhance the overall global time series forecasting model performance. Finally, to test the effectiveness of the proposed approach, it was validated on two publicly available real-world datasets.

show abstract

EXAD: A System for Explainable Anomaly Detection on Big Data Traces

Cited by 23 publications

References 13 publications

Anomaly explanation: A review

Anomaly explanation: A review

A survey on outlier explanations

Explaining Bad Forecasts in Global Time Series Models

Contact Info

Product

Resources

About