Factor Analysis of Mixed Data for Anomaly Detection

Davidow, Matthew; Matteson, David S.

doi:10.48550/arxiv.2005.12129

Cited by 1 publication

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One is from the UCI machine learning repository (Dua and Graff, 2017), and three from the ODDS anomaly detection repository Descriptions of these datasets are presented in the appendix. We apply PCA-like anomaly detection algorithms, where some of the algorithms make use of kurtosis, which has been shown to increase anomaly detection performance across several domains, see the appendix for more explanation of these algorithms (Davidow and Matteson, 2020). We expect the kurtosis methods to be more accurate and similar to each other, as they extract similar features.…”

Section: Methodsmentioning

confidence: 99%

“…The second type of anomaly is separated by the largest principal components, since these anomalies contribute to these components making them the largest. We use a set of four anomaly detection algorithms that make use the first principal components, and a set of four anomaly detection algorithms that make use of the last principal components, as described in (Davidow, 2020).…”

Section: Two Anom Dataset Descriptionmentioning

confidence: 99%

“…We show summary statistics of these datsets in table 3. We apply PCA-like anomaly detection algorithms, where some of the algorithms make use of kurtosis, which has been shown to be more accurate and similar to each other, as they extract similar features which has been shown to increase anomaly detection performance across several domains (Davidow, 2020).…”

Section: Experimental Datasetsmentioning

confidence: 99%

See 2 more Smart Citations

Copula Quadrant Similarity for Anomaly Scores

Davidow,

Matteson

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Practical anomaly detection requires applying numerous approaches due to the inherent difficulty of unsupervised learning. Direct comparison between complex or opaque anomaly detection algorithms is intractable; we instead propose a framework for associating the scores of multiple methods. Our aim is to answer the question: how should one measure the similarity between anomaly scores generated by different methods? The scoring crux is the extremes, which identify the most anomalous observations. A pair of algorithms are defined here to be similar if they assign their highest scores to roughly the same small fraction of observations. To formalize this, we propose a measure based on extremal similarity in scoring distributions through a novel upper quadrant modeling approach, and contrast it with tail and other dependence measures. We illustrate our method with simulated and real experiments, applying spectral methods to cluster multiple anomaly detection methods and to contrast our similarity measure with others. We demonstrate that our method is able to detect the clusters of anomaly detection algorithms to achieve an accurate and robust ensemble algorithm.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Two Anom Dataset Descriptionmentioning

confidence: 99%

Section: Experimental Datasetsmentioning

confidence: 99%

See 1 more Smart Citation