A Semisupervised Approach to the Detection and Characterization of Outliers in Categorical Data

Ienco, Dino; Pensa, Ruggero G.; Meo, Rosa

doi:10.1109/tnnls.2016.2526063

Cited by 41 publications

(19 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…To apply the labeled data to detect anomalies, some semisupervised methods [16]- [21] have been proposed. However, these methods focus on learning the behavior patterns of normal data, which require a large amount of labeled normal data for training.…”

Section: B Deep Anomaly Detectionmentioning

confidence: 99%

“…If the information contained in these labeled data can be effectively applied, the accuracy of anomaly detection can be well improved. Therefore, some so-called semisupervised anomaly detection methods [16]- [21] using labeled data have been proposed. However, these methods can only use a large amount of labeled normal data for training, which generates extremely high labeling costs.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ConNet: Deep Semi-Supervised Anomaly Detection Based on Sparse Positive Samples

Gao

Cheng

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Existing semi-supervised anomaly detection methods usually use a large amount of labeled normal data for training, which have the problem of high labeling costs. Only a few semi-supervised methods utilize unlabeled data and a few labeled anomalies to train models. However, these kinds of methods usually encounter two problems: (i) since anomalies usually have different behavior patterns or the internal mechanisms that produce anomalies are complex and diverse, a few labeled anomalies cannot cover all anomaly types; and (ii) the amount of unlabeled data in the training set is substantially greater than the amount of labeled data, which leads to that unlabeled data with contamination often dominates the training process. To solve these two problems, we propose the semi-supervised anomaly detection method named ConNet and a new loss function named concentration loss. Specifically, ConNet consists of two stages. Firstly, we obtain the prior anomaly score of unlabeled data via prior estimation module and attach the prior anomaly score to unlabeled data as the training weight. Then, an anomaly scoring network is training to assign anomaly scores to data instances, which can ensure that the anomaly scores of anomalies significantly deviate from those of normal instances. We have conducted experiments on thirteen real-world data sets and tested the performance of our method in terms of detection accuracy, utilization efficiency of labeled data, and robustness to different contamination rates. The experimental results show that the performance of our method is significantly better than those of the state-of-the-art anomaly detection methods.

show abstract

Section: B Deep Anomaly Detectionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

ConNet: Deep Semi-Supervised Anomaly Detection Based on Sparse Positive Samples

Gao

Cheng

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Depending on the availability of class labels, there are supervised, semi-supervised, and unsupervised outlier detection methods [8]. Outlier detection methods can also be grouped into different categories [9] like statistical methods, proximity based methods, classification based methods and clustering based methods, depending on the assumptions made on outliers versus normal data.…”

Section: Fig1 Objects Outside the Ellipse Are Outliers And Objectsmentioning

confidence: 99%

Correlation and Probability Based Similarity Measure for Detecting Outliers in Categorical Data

Thomas*,

Judith

2020

IJITEE

View full text Add to dashboard Cite

Determining the similarity or distance among data objects is an important part in many research fields such as statistics, data mining, machine learning etc. There are many measures available in the literature to define the distance between two numerical data objects. It is difficult to define such a metric to measure the similarity between two categorical data objects since categorical data objects are not ordered. Only a few distance measures are available in the literature to find the similarities among categorical data objects. This paper presents a comparative evaluation of various similarity measures for categorical data and also introduces a novel similarity measure for categorical data based on occurrence frequency and correlation. We evaluated the performance of these similarity measures in the context of outlier detection task in data mining using real world data sets. Experimental results show that the proposed similarity measure outperform the existing similarity measures to detect outliers in categorical datasets. The performances are evaluated in the context of outlier detection task in data mining

show abstract

“…Li et al [20] proposed a data-analysis method for modeling and predicting the daily electricity consumption in buildings, which can also be used to forecast and detect of abnormal energy use. Ienco et al [21] analyzed the relationship among features to extract a discriminative characterization for anomalous instances. Fiore et al [22] used Discriminative Restricted Boltzmann Machine to combine the expressive power of generative models and infer knowledge from incomplete training data.…”

Section: Related Workmentioning

confidence: 99%