Issues and Advances in Anomaly Detection Evaluation for Joint Human-Automated Systems

Rieth, Cory A.; Amsel, Ben D.; Tran, Randy; Cook, Maia B.

doi:10.1007/978-3-319-60384-1_6

Cited by 20 publications

(28 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section, the methodology proposed in Section IV is applied to validate it in an industrial scenario. To validate the methodology, we used the dataset generated by Rieth et al [31] using the TE process, which is a simulated testbed of a chemical process where authors introduced 20 anomalies. The authors published four files: training and test files with anomalies and training and test files free of anomalies.…”

Section: Methodology Validationmentioning

confidence: 99%

A Methodology for Evaluating the Robustness of Anomaly Detectors to Adversarial Attacks in Industrial Scenarios

et al. 2022

View full text Add to dashboard Cite

Anomaly Detection systems based on Machine and Deep learning are the most promising solutions to detect cyberattacks in the industry. However, these techniques are vulnerable to adversarial attacks that downgrade predictions performance. Several techniques haven been proposed to measure the robustness of Anomaly Detection in the literature. However, they do not consider that, although a small perturbation in an anomalous sample belonging to an attack, i.e., Denial of Service, could cause it to be misclassified as normal while retaining its ability to damage, an excessive perturbation might also transform it into a truly normal sample, with no real impact on the industrial system. This paper presents a methodology to calculate the robustness of Anomaly Detection models in industrial scenarios. The methodology comprises four steps and use a set of additional models called support models to determine if an adversarial sample remains anomalous. We carried out the validation using the Tennessee Eastman process, a simulated testbed of a chemical process. In such scenario, we applied the methodology to both a Long-Short Term Memory (LSTM) neural network and 1-dimensional Convolutional Neural Network (1D-CNN) focused on detecting anomalies produced by different cyberattacks. The experiments showed that 1D-CNN is significantly more robust than LSTM for our testbed. Specifically, a perturbation of 60% (empirical robustness of 0.6) of the original sample is needed to generate adversarial samples for LSTM, whereas in 1D-CNN the perturbation required increases up to 111% (empirical robustness of 1.11).INDEX TERMS adversarial attacks, evasion attacks, industrial control systems, machine learning, deep learning, robustness I. INTRODUCTION

show abstract

Section: Methodology Validationmentioning

confidence: 99%

A Methodology for Evaluating the Robustness of Anomaly Detectors to Adversarial Attacks in Industrial Scenarios

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In this study, we demonstrate root cause identification using the proposed methodology for process faults in the Tennessee Eastman process. For this, we used the simulated data provided by Rieth, Amsel, Tran and Cook (2018)…”

Section: Case Study : Tennessee Eastman Processmentioning

confidence: 99%

Bayesian network-based root cause analysis and fault pathway identification in complex industrial processes

et al. 2021

View full text Add to dashboard Cite

Real-time root cause identification (RCI) of faults or abnormal events in industries gives plant personnel the opportunity to address the faults before they progress and lead to failure. RCI in industrial systems must deal with their complex behavior, variable interactions, corrective actions of control systems and variability in faulty behavior. Bayesian networks (BNs) is a data-driven graph-based method that utilizes multivariate sensor data generated during industrial operations for RCI. Bayesian networks, however, require data discretization if data contains both discrete and continuous variables. Traditional discretization techniques such as equal width (EW) or equal frequency (EF) discretization result in loss of dynamic information and often lead to erroneous RCI. To deal with this limitation, we propose the use of a dynamic discretization technique called Bayesian Blocks (BB) which adapts the bin sizes based on the properties of data itself. In this work, we compare the effectiveness of three discretization techniques, namely EW, EF and BB coupled with Bayesian Networks on generation of fault propagation (causal) maps and root cause identification in complex industrial systems. We demonstrate the performance of the three methods on the industrial benchmark Tennessee-Eastman (TE) process. For two complex faults in the TE process, the BB with BN method successfully diagnosed correct root causes of the faults, and reduced redundancy (up to 50%) and improved the propagation paths in causal maps compared to other two techniques.

show abstract

“…It includes single set of simulation for normal and 21 faulty operations separately (yielding 22 simulation datasets). The latter is taken from the study by Rieth et al having measurements from normal and first 20 faults provided in Table . It involves 500 set of simulations for normal and 20 faulty operations separately (yielding 10,500 simulation datasets).…”

Section: Tennessee Eastman Process: Model and Datasetmentioning

confidence: 99%

“…Thus, training sets consists of 500 samples, whereas test sets contain 960 samples per set of simulation. Further information on the process and simulation can be found in other references .…”

Section: Tennessee Eastman Process: Model and Datasetmentioning

confidence: 99%

A nonlinear support vector machine‐based feature selection approach for fault detection and diagnosis: Application to the Tennessee Eastman process

2019

View full text Add to dashboard Cite

In this article, we present (1) a feature selection algorithm based on nonlinear support vector machine (SVM) for fault detection and diagnosis in continuous processes and (2) results for the Tennessee Eastman benchmark process. The presented feature selection algorithm is derived from the sensitivity analysis of the dual C‐SVM objective function. This enables simultaneous modeling and feature selection paving the way for simultaneous fault detection and diagnosis, where feature ranking guides fault diagnosis. We train fault‐specific two‐class SVM models to detect faulty operations, while using the feature selection algorithm to improve the accuracy and perform the fault diagnosis. Our results show that the developed SVM models outperform the available ones in the literature both in terms of detection accuracy and latency. Moreover, it is shown that the loss of information is minimized with the use of feature selection techniques compared to feature extraction techniques such as principal component analysis (PCA). This further facilitates a more accurate interpretation of the results. © 2018 American Institute of Chemical Engineers AIChE J, 65: 992–1005, 2019

show abstract

Issues and Advances in Anomaly Detection Evaluation for Joint Human-Automated Systems

Cited by 20 publications

References 23 publications

A Methodology for Evaluating the Robustness of Anomaly Detectors to Adversarial Attacks in Industrial Scenarios

A Methodology for Evaluating the Robustness of Anomaly Detectors to Adversarial Attacks in Industrial Scenarios

Bayesian network-based root cause analysis and fault pathway identification in complex industrial processes

A nonlinear support vector machine‐based feature selection approach for fault detection and diagnosis: Application to the Tennessee Eastman process

Contact Info

Product

Resources

About