On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study

Campos, Guilherme Oliveira; Zimek, Arthur; Sander, Jöerg; Campello, Ricardo J. G. B.; Micenková, Barbora; Schubert, Erich; Assent, Ira; Houle, Michael E.

doi:10.1007/s10618-015-0444-8

Cited by 571 publications

(403 citation statements)

References 60 publications

Supporting

Mentioning

367

Contrasting

Unclassified

Order By: Relevance

“…We see three reasons why studying the issue is necessary, namely (1) increasing the reliability of critical infrastructures, (2) coping with attacks, and (3) systematic evaluation of outlier detection algorithms. We now elaborate on these points one by one.…”

Section: Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Hiding outliers in high-dimensional data spaces

Steinbuß

Böhm

2017

Int J Data Sci Anal

View full text Add to dashboard Cite

Detecting outliers in high-dimensional data is crucial in many domains. Due to the curse of dimensionality, one typically does not detect outliers in the full space, but in subspaces of it. More specifically, since the number of subspaces is huge, the detection takes place in only some subspaces. In consequence, one might miss hidden outliers, i.e., outliers only detectable in certain subspaces. In this paper, we take the opposite perspective, which is of practical relevance as well, and study how to hide outliers in high-dimensional data spaces. We formally prove characteristics of hidden outliers. We also propose an algorithm to place them in the data. It focuses on the regions close to existing data objects and is more e cient than an exhaustive approach. In experiments, we both evaluate our formal results and show the usefulness of our algorithm using di↵erent subspace selection schemes, outlier detection methods and data sets.

show abstract

Section: Motivationmentioning

confidence: 99%

“…Current evaluation schemes often either use an already existing minority class as outlying or downsample the data to one rare class [2]. However, these outliers are not necessarily subspace outliers, in contrast to hidden outliers generated with our approach.…”

Section: Evaluation Of Subspace Outlier Detectionmentioning

confidence: 99%

Hiding outliers in high-dimensional data spaces

Steinbuß

Böhm

2017

Int J Data Sci Anal

View full text Add to dashboard Cite

show abstract

“…Evaluation is from a scientific point of view not less but more important than the design of ever newer and "better" (really?) methods, and is typically challenging and far from trivial, as has been discussed for other areas of data mining as well [1,2,7,9,20,22].…”

Section: So What?mentioning

confidence: 99%

Redundancies in Data and their Effect on the Evaluation of Recommendation Systems: A Case Study on the Amazon Reviews Datasets

Basaran

Ntoutsi

Zimek³

2017

Proceedings of the 2017 SIAM International Conference on Data Mining

Self Cite

View full text Add to dashboard Cite

A collection of datasets crawled from Amazon, "Amazon reviews", is popular in the evaluation of recommendation systems. These datasets, however, contain redundancies (duplicated recommendations for variants of certain items). These redundancies went unnoticed in earlier use of these datasets and thus incurred to a certain extent wrong conclusions in the evaluation of algorithms tested on these datasets. We analyze the nature and amount of these redundancies and their impact on the evaluation of recommendation methods. While the general and obvious conclusion is that redundancies should be avoided and datasets should be carefully preprocessed, we observe more specifically that their impact depends on the complexity of the methods. With this work, we also want to raise the awareness of the importance of data quality, model understanding, and appropriate evaluation.

show abstract

“…Approximate entropy is a statistical method devised to measure complexity and regularity of a system, which has proven to be suitable for short (but more than 100 data points) and noisy time-series [27]. 5 H a reflects the existence of patterns in a series that makes future values more predictable. Small H a values tending to 0 are expected for time series that contain repetitive patterns, whereas high values are expected for chaotic behaviours.…”

Section: Regularity/predictability Estimatorsmentioning

confidence: 99%

Analytic Study of Features for the Detection of Covert Timing Channels in NetworkTraffic

V�zquez¹,

Annessi

Zseby

2017

JCSM

View full text Add to dashboard Cite

Covert timing channels are security threats that have concerned the expert community from the beginnings of secure computer networks. In this paper we explore the nature of covert timing channels by studying the behavior of a selection of features used for their detection. Insights are obtained from experimental studies based on ten covert timing channels techniques published in the literature, which include popular and novel approaches. The study digs into the shapes of flows containing covert timing channels from a statistical perspective as well as using supervised and unsupervised machine learning algorithms. Our experiments reveal which features are recommended for building detection methods and draw meaningful representations to understand the problem space. Covert timing channels show high histogramdistance based outlierness, but insufficient to clearly discriminate them from normal traffic. On the other hand, traffic features do show dependencies that allow separating subspaces and facilitate the identification of covert timing channels. The conducted study shows the detection difficulties due to the high shape variability of normal traffic and suggests the implementation of semi-supervised techniques to develop accurate and reliable detectors.

show abstract

On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study

Cited by 571 publications

References 60 publications

Hiding outliers in high-dimensional data spaces

Hiding outliers in high-dimensional data spaces

Redundancies in Data and their Effect on the Evaluation of Recommendation Systems: A Case Study on the Amazon Reviews Datasets

Analytic Study of Features for the Detection of Covert Timing Channels in NetworkTraffic

Contact Info

Product

Resources

About