A relevant subspace based contextual outlier mining algorithm

Zhang, Jifu; Yu, Xiaolong; Li, Yonghong; Zhang, Sulan; Xun, Yaling; Qin, Xiao

doi:10.1016/j.knosys.2016.01.013

Cited by 33 publications

(10 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For subspace learning-based outlier methods, the key is to find the relevant outliers by sifting through different subsets of dimensions in the data in an ordered way. Generally, these methods can be divided into two categories: the sparse subspace methods 17,18,19 and the relevant subspace methods 20,21,22,23,24,25,26 .…”

Section: Subspace Learning-based Methodsmentioning

confidence: 99%

A Comparison of Outlier Detection Techniques for High-Dimensional Data

Xu¹,

Liu²,

Li³

et al. 2018

IJCIS

View full text Add to dashboard Cite

Outlier detection is a hot topic in machine learning. With the newly emerging technologies and diverse applications, the interest of outlier detection is increasing greatly. Recently, a significant number of outlier detection methods have been witnessed and successfully applied in a wide range of fields, including medical health, credit card fraud and intrusion detection. They can be used for conventional data analysis. However, it is not a trivial work to identify rare behaviors or patterns out from complicated data. In this paper, we provide a brief overview of the outlier detection methods for high-dimensional data, and offer comprehensive understanding of the-state-of-the-art techniques of outlier detection for practitioners. Specifically, we firstly summarize the recent advances on outlier detection for high-dimensional data, and then make an extensive experimental comparison to the popular detection methods on public datasets. Finally, several challenging issues and future research directions are discussed.

show abstract

Section: Subspace Learning-based Methodsmentioning

confidence: 99%

A Comparison of Outlier Detection Techniques for High-Dimensional Data

Xu¹,

Liu²,

Li³

et al. 2018

IJCIS

View full text Add to dashboard Cite

show abstract

“…An alternative to use neighbor-based methods is to detect anomalies taken into account that they are grouped in a zone of the data space. Thus, the anomaly detection problem is solved as a subspace learning problem [33][34][35][36]. Although this method work well in some cases, finding the number of subspaces in which the anomalies are distributed is not trivial.…”

Section: Related Workmentioning

confidence: 99%

Deep autoencoder for false positive reduction in handgun detection

Vállez

Velasco-Mata

Déniz

2020

Neural Comput & Applic

View full text Add to dashboard Cite

In an object detection system, the main objective during training is to maintain the detection and false positive rates under acceptable levels when the model is run over the test set. However, this typically translates into an unacceptable rate of false alarms when the system is deployed in a real surveillance scenario. To deal with this situation, which often leads to system shutdown, we propose to add a filter step to discard part of the new false positive detections that are typical of the new scenario. This step consists of a deep autoencoder trained with the false alarm detections generated after running the detector over a period of time in the new scenario. Therefore, this step will be in charge of determining whether the detection is a typical false alarm of that scenario or whether it is something anomalous for the autoencoder and, therefore, a true detection. In order to decide whether a detection must be filtered, three different approaches have been tested. The first one uses the autoencoder reconstruction error measured with the mean squared error to make the decision. The other two use the k-NN (k-nearest neighbors) and one-class SVMs (support vector machines) classifiers trained with the autoencoder vector representation. In addition, a synthetic scenario has been generated with Unreal Engine 4 to test the proposed methods in addition to a dataset with real images. The results obtained show a reduction in the number of false positives between 22.5% and 87.2% and an increase in the system’s precision of 1.2%$$-47$$ - 47 % when the autoencoder is applied.

show abstract

“…Based on the above characteristics of data with doublepeaked emission lines, data preprocessing is worked out before experiments. A characteristic extraction method based on relevant subspace (RS) is used to obtain the characteristics related with double peaks [1], [36], [58]. A dataset including 345 spectra with double-peaked emission lines, which are known and identified currently, is selected from LAMOST DR5 to extract the useful characteristics [14], [15].…”

Section: Experiments Analysis a Data Preprocessingmentioning

confidence: 99%

SVM-Lattice: A Recognition and Evaluation Frame for Double-Peaked Profiles

Yang

Cai

et al. 2020

IEEE Access

Self Cite

View full text Add to dashboard Cite

In big data era, the special data with rare characteristics may be of great significations. However, it is very difficult to automatically search these samples from the massive and high-dimensional datasets and systematically evaluate them. The DoPS, our previous work [1], provided a search method of rare spectra with double-peaked profiles from massive and high-dimensional data of LAMOST survey. The identification of the results is mainly depended on visually inspection by astronomers. In this paper, as a follow-up study, a new lattice structure named SVM-Lattice is designed based on SVM(Support Vector Machine) and FCL(Formal Concept Lattice) and particularly applied in the recognition and evaluation of rare spectra with double-peaked profiles. First, each node in the SVM-Lattice structure contains two components: the intents are defined by the support vectors trained by the spectral samples with the specific characteristics, and the relevant extents are all the positive samples classified by the support vectors. The hyperplanes can be extracted from every lattice node and used as classifiers to search targets by categories. A generalization and specialization relationship is expressed between the layers, and higher layers indicate higher confidence of targets. Then, including a SVM-Lattice building algorithm, a pruning algorithm based on association rules, and an evaluation algorithm, the supporting algorithms are provided and analysed. Finally, for the recognition and evaluation of spectra with double-peaked profiles, several data sets from LAMOST survey are used as experimental dataset. The results exhibit good consistency with traditional methods, more detailed and accurate evaluations of classification results, and higher searching efficiency than other similar methods. INDEX TERMS SVM-Lattice, double-peaked profiles, support vector machine, formal concept lattice.

show abstract

A relevant subspace based contextual outlier mining algorithm

Cited by 33 publications

References 24 publications

A Comparison of Outlier Detection Techniques for High-Dimensional Data

A Comparison of Outlier Detection Techniques for High-Dimensional Data

Deep autoencoder for false positive reduction in handgun detection

SVM-Lattice: A Recognition and Evaluation Frame for Double-Peaked Profiles

Contact Info

Product

Resources

About