Detecting outlying properties of exceptional objects

Angiulli, Fabrizio; Fassetti, Fabio; Palopoli, Luigi

doi:10.1145/1508857.1508864

Cited by 75 publications

(54 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subspace clustering [3,7,53,30,44] Projected clustering [2,41] (data descr.) SVDD [55], SSSVDD [18] (rare category) RACH [21] (rare category) PALM [20] Knorr and Ng [27] RefOUT [24], CP [34] LODI [11], LOGP [10] EXPREX [5] (Explaining black box classifiers) LIME [51] EXstream [58] Explainer [50] SRF [29], Krimp [56], RuleFit [15], Ripper [9] x-PACS [this paper] Extending earlier work [4] on explaining single outliers, [5] aims to explain groups of outlier points or what they call sub-populations. They search for context, feature pairs, where the (single) feature can differentiate as many outliers as possible from normal points that share the same context.…”

Section: Related Workmentioning

confidence: 99%

Explaining anomalies in groups with characterizing subspace rules

Macha

Akoglu

2018

Data Min Knowl Disc

View full text Add to dashboard Cite

Anomaly detection has numerous applications and has been studied vastly. We consider a complementary problem that has a much sparser literature: anomaly description. Interpretation of anomalies is crucial for practitioners for sense-making, troubleshooting, and planning actions. To this end, we present a new approach called x-PACS (for eXplaining Patterns of Anomalies with Characterizing Subspaces), which "reverse-engineers" the known anomalies by identifying (1) the groups (or patterns) that they form, and (2) the characterizing subspace and feature rules that separate each anomalous pattern from normal instances. Explaining anomalies in groups not only saves analyst time and gives insight into various types of anomalies, but also draws attention to potentially critical, repeating anomalies. In developing x-PACS, we first construct a desiderata for the anomaly description problem. From a descriptive data mining perspective, our method exhibits five desired properties in our desiderata. Namely, it can unearth anomalous patterns (i) of multiple different types, (ii) hidden in arbitrary subspaces of a high dimensional space, (iii) interpretable by human analysts, (iv) different from normal patterns of the data, and finally (v) succinct, providing a short data description. No existing work on anomaly description satisfies all of these properties simultaneously. Furthermore, x-PACS is highly parallelizable; it is linear on the number of data points and exponential on the (typically small) largest characterizing subspace size. The anomalous patterns that x-PACS finds constitute interpretable "signatures", and while it is not our primary goal, they can be used for anomaly detection. Through extensive experiments on real-world datasets, we show the effectiveness and superiority of x-PACS in anomaly explanation over various baselines, and demonstrate its competitive detection performance as compared to the state-of-the-art.

show abstract

Section: Related Workmentioning

confidence: 99%

Explaining anomalies in groups with characterizing subspace rules

Macha

Akoglu

2018

Data Min Knowl Disc

View full text Add to dashboard Cite

show abstract

“…Hautamaki et al 2004;Angiulli et al 2009) or clustering-based techniques (e.g. Yang & Wang 2003;Basu et al 2004).…”

Section: Novelty Detectionmentioning

confidence: 99%

Automated novelty detection in the WISE survey with one-class support vector machines

Solarz

Bilicki

Gromadzki

et al. 2017

A&A

View full text Add to dashboard Cite

Wide-angle photometric surveys of previously uncharted sky areas or wavelength regimes will always bring in unexpected sourcesnovelties or even anomalies -whose existence and properties cannot be easily predicted from earlier observations. Such objects can be efficiently located with novelty detection algorithms. Here we present an application of such a method, called one-class support vector machines (OCSVM), to search for anomalous patterns among sources preselected from the mid-infrared AllWISE catalogue covering the whole sky. To create a model of expected data we train the algorithm on a set of objects with spectroscopic identifications from the SDSS DR13 database, present also in AllWISE. The OCSVM method detects as anomalous those sources whose patterns -WISE photometric measurements in this case -are inconsistent with the model. Among the detected anomalies we find artefacts, such as objects with spurious photometry due to blending, but more importantly also real sources of genuine astrophysical interest. Among the latter, OCSVM has identified a sample of heavily reddened AGN/quasar candidates distributed uniformly over the sky and in a large part absent from other WISE-based AGN catalogues. It also allowed us to find a specific group of sources of mixed types, mostly stars and compact galaxies. By combining the semi-supervised OCSVM algorithm with standard classification methods it will be possible to improve the latter by accounting for sources which are not present in the training sample, but are otherwise well-represented in the target set. Anomaly detection adds flexibility to automated source separation procedures and helps verify the reliability and representativeness of the training samples. It should be thus considered as an essential step in supervised classification schemes to ensure completeness and purity of produced catalogues.

show abstract

“…Again, the notions of context, reference groups and outlier groups are not modeled simultaneously in OC 3 . Angiulli et al [2] studied a related by orthogonal problem. Given a multidimensional database and a query object in the database, find the top-k subset of attributes that the query object receives the highest outlier score.…”

Section: Outlier-idmentioning

confidence: 99%

Mining multidimensional contextual outliers from categorical relational data

Tang

Pei

Bailey

et al. 2015

IDA

View full text Add to dashboard Cite

A wide range of methods have been proposed for detecting different types of outliers in full space and subspaces. However, the interpretability of outliers, that is, explaining in what ways and to what extent an object is an outlier, remains a critical open issue. In this paper, we develop a notion of contextual outliers on categorical data. Intuitively, a contextual outlier is a small group of objects that share strong similarity with a significantly larger reference group of objects on some attributes, but deviate dramatically on some other attributes. We develop a detection algorithm, and conduct experiments to evaluate our approach.

show abstract

Detecting outlying properties of exceptional objects

Cited by 75 publications

References 25 publications

Explaining anomalies in groups with characterizing subspace rules

Explaining anomalies in groups with characterizing subspace rules

Automated novelty detection in the WISE survey with one-class support vector machines

Mining multidimensional contextual outliers from categorical relational data

Contact Info

Product

Resources

About