Improved autoencoder for unsupervised anomaly detection

Cheng, Zhen; Wang, Siwei; Zhang, Pei; Wang, Siqi; Liu, Xinwang; Zhu, En

doi:10.1002/int.22582

Cited by 55 publications

(25 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…. In order to investigate the detection performance of the proposed algorithm for time-frequency overlapped interference, one method based on improved autoencoder (IAE) [28] and another method based on improved generative adversarial network (IGAN) [29] are compared in this paper, which are commonly used for anomaly detection in deep learning.…”

Section: Resultsmentioning

confidence: 99%

A Novel Approach Based on Generative Adversarial Network for Interference Detection in Wireless Communications

Zhu

Yao

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the rapid growth of wireless devices, the communication environment gets complex. The detection of interference or unauthorized signals can improve spectrum efficiency, which is a key technology for limited spectrum resources. Traditional detection methods analyze the parameter characteristics of the received signal. But it is difficult to detect interference with the same time and frequency as the original signal by those feature engineering. As a classical problem in deep learning, anomaly detection is usually solved by supervised learning. But a more challenging situation is to detect unknown or invisible anomalies. It means that the number of abnormal samples is insufficient and the data is highly biased toward the normal samples. In this paper, a wireless communication interference detection algorithm based on generative adversarial network (GAN) is proposed. In the semi-supervised learning scenario, the algorithm detects the time-frequency overlapped interference by the reconstruction strategy. The generator adopts the encoder-decoder-encoder architecture. In the training process, the model jointly learns the data distribution of normal samples by minimizing the distance in both the signal space and the latent space. In the inference phase, a large distance metric implies an abnormal sample. Experiments on simulated communication datasets show the superiority of the proposed algorithm.

show abstract

Section: Resultsmentioning

confidence: 99%

A Novel Approach Based on Generative Adversarial Network for Interference Detection in Wireless Communications

Zhu

Yao

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…This contribution mainly investigates the use of the autoencoder (AE) for novelty detection [2,11]. The AE is a type of artificial neural network that learns the data representation in an unsupervised manner.…”

Section: Autoencodermentioning

confidence: 99%

Autoencoder and Mahalanobis distance for novelty detection in structural health monitoring data of an offshore wind turbine.

Weil

Weijtjens

Devriendt

2022

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

Structural Health Monitoring (SHM) has seen an explosion in data gathering in the last few years. This is illustrated in the offshore wind industry through an increase in the amount of placed offshore wind turbines (OWT), a higher rate of SHM instrumented OWTs and an increase in the sampling rate. The growing data gathering has led to the interest of big data techniques in the SHM industry. This paper introduces a new more robust unsupervised novelty detection pipeline combining an autoencoder and the Mahalanobis distance and comparing this combination to both methods separately. This reduces the high dimensional input to a one dimensional novelty index for detecting anomalies in the OWT. Additionally the research considers the challenges that the downtime of non operationally essential sensors poses, a method is introduced to guarantee high model availability without losing the benefit of high fidelity.

show abstract

“…Details are discussed in the following sections. (1) and ℓ (2) increases the total GCD do ⊳ ℓ (1) and ℓ (2) as in Equation ( 21) 32:…”

Section: Kernel-based Online Dependence Clusteringmentioning

confidence: 99%

“…If merging the two candidates, ℓ (1) and ℓ (2) , increases the total GCD, the two clusters become one, and the number of clusters decreases to l − 1. For every iteration, the merger step searches for two clusters with the smallest separation and determines whether they can be merged to increase the total GCD.…”

Section: Classifier Updatementioning

confidence: 99%

“…With the recent developments in information and communication technologies, an increasing amount of data is produced in the form of streams. 1 Practically, the data streams are generated and propagated in an automated fashion by various application sources, such as network monitoring, intrusion detection, web click streams, and numerous types of sensor systems. 2 Analyzing such data has been an active research topic in computer science, machine learning, and data mining.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online dependence clustering of multivariate streaming data using one‐class SVMs

Lee

2021

Int J of Intelligent Sys

View full text Add to dashboard Cite

Online clustering of multivariate streaming data has attracted considerable interest in recent years due to the abundance of data sources. Numerous studies in this field have been performed, but they usually suffer from the practical problems associated with discovering arbitrary‐shaped clusters, specifying major parameters in advance, and detecting aberrant observations. Addressing these issues is important for online‐clustering tasks, where data arrive in continuous streams and group behaviors change simultaneously. In this paper, we propose a kernel‐based online dependence clustering, namely, KODC, that not only estimates the cluster membership using one‐class support vector machines (OC‐SVMs), but also detects outliers distant from the identified clusters by aggregating OC‐SVM decisions in a realtime basis. At the base level, we use a new measure of connective dependence that forms the graph connected via modified Markovian transitions to enable large‐scale clustering. The proposed framework introduces the coherence threshold to extract data points, which can represent a cluster to which they belong, thus controlling the computational complexity without degrading the clustering performance. To track the pattern evolution over time, KODC also updates the classifier configuration maximizing the total group connective dependence. We evaluate this framework on both several synthetic and real‐world data sets involving multivariate streaming data, and compare it experimentally with other popular online‐clustering methods in terms of four evaluation metrics. The results show that our framework effectively identifies the clusters and outliers, especially in various shaped data subject to change over time, without requiring any prior knowledge of the data.

show abstract

Improved autoencoder for unsupervised anomaly detection

Cited by 55 publications

References 24 publications

A Novel Approach Based on Generative Adversarial Network for Interference Detection in Wireless Communications

A Novel Approach Based on Generative Adversarial Network for Interference Detection in Wireless Communications

Autoencoder and Mahalanobis distance for novelty detection in structural health monitoring data of an offshore wind turbine.

Online dependence clustering of multivariate streaming data using one‐class SVMs

Contact Info

Product

Resources

About