A One-Class Kernel Fisher Criterion for Outlier Detection

Abstract

Recently, Dufrenois and Noyer proposed a one class Fisher's linear discriminant to isolate normal data from outliers. In this paper, a kernelized version of their criterion is presented. Originally on the basis of an iterative optimization process, alternating between subspace selection and clustering, I show here that their criterion has an upper bound making these two problems independent. In particular, the estimation of the label vector is formulated as an unconstrained binary linear problem (UBLP) which can be solved using an iterative perturbation method. Once the label vector is estimated, an optimal projection subspace is obtained by solving a generalized eigenvalue problem. Like many other kernel methods, the performance of the proposed approach depends on the choice of the kernel. Constructed with a Gaussian kernel, I show that the proposed contrast measure is an efficient indicator for selecting an optimal kernel width. This property simplifies the model selection problem which is typically solved by costly (generalized) cross-validation procedures. Initialization, convergence analysis, and computational complexity are also discussed. Lastly, the proposed algorithm is compared with recent novelty detectors on synthetic and real data sets.

Structural Minimax Probability Machine

“…We believe that the structural information also exists in other learning tasks (e.g., one-class learning [15], [33], ordinalclass learning [31], [34], multiclass learning [35], [36], and so on). In the future, we plan to extend SMPM to one-class learning [15], [33], ordinal-class learning [31], [34], multiclass learning [35], [36], and other learning tasks. Although the binary search procedure can solve SMPM effectively, the running time is not fast enough especially when the training data size (or the feature size) is large.…”

Robust One-Class Kernel Spectral Regression

“…In [31], an incremental version of the method in [27] is proposed to increase computational efficiency. A generalised Rayleigh quotient specifically designed for outlier detection is presented in [28], [36] where the method tries to find an optimal hyperplane which is closest to the target data and farthest from the outliers utilising two scatter matrices corresponding to the outliers and target data. In [36], the generalised eigenvalue problem is replaced by an approximate conjugate gradient solution to moderate the computational cost of the method in [28].…”

“…A generalised Rayleigh quotient specifically designed for outlier detection is presented in [28], [36] where the method tries to find an optimal hyperplane which is closest to the target data and farthest from the outliers utilising two scatter matrices corresponding to the outliers and target data. In [36], the generalised eigenvalue problem is replaced by an approximate conjugate gradient solution to moderate the computational cost of the method in [28]. A later study [37] tries to address limitations of the method in [28], [36] in terms of the availability of outlier samples and difference in the densities of target and nontarget observations via a null-space approach.…”

“…Among well-known instances of the generative approaches are the parametric and non-parametric density estimation methods [13], [14], [15], neural-network based approaches [16], [17], sparse representation methods [18], [19], etc. Notable representatives of the non-generative methods include support vectorbased approaches [20], [21], convex hull methods [22], [23], cluster approaches [24] and subspace techniques [25], [26], [27], [28], [29]. As a result of the particular importance given to classification, rather than generative process modelling, the non-generative methods are typically believed to yield better classification performance.…”

One-Class Classifiers Based on Entropic Spanning Graphs

“…The analysis of large volumes of data is hampered by many technical problems, including the ones related to the quality and interpretation of associated information. One-class classifier design is an important research endeavour [1], [2] that can be used to tackle problems of anomaly/novelty detection or, more generally, to recognize outliers in incoming data [3]- [8]. Several different methods have been proposed in the literature, including clustering-based techniques, kernel methods, and statistical approaches (see [9] for a recent survey).…”