Ellipsoidal Subspace Support Vector Data Description

Abstract

In this paper, we propose a novel method for transforming data into a low-dimensional space optimized for one-class classification. The proposed method iteratively transforms data into a new subspace optimized for ellipsoidal encapsulation of target class data. We provide both linear and non-linear formulations for the proposed method. The method takes into account the covariance of the data in the subspace; hence, it yields a more generalized solution as compared to the data description in the subspace by hyperspherical encapsulation of target class data. We propose different regularization terms expressing the class variance in the projected space. We compare the results with classic and recently proposed one-class classification methods and achieve competing results and show clear improvement compared to the other support vector based methods. The proposed method is also noticed to converge much faster than recently proposed Subspace Support Vector Data Description.

Subspace Support Vector Data Description

“…One-class classification methods can broadly be divided into two categories 1) support vector (SV)-based methods [12], [21] 2) non-SV-based methods [22], [23]. In SV-based methods, a decision boundary represented by so-called support vectors is formed by solving an optimization problem.…”

“…The projection matrix is orthogonalized after every update. Recently, a more general Ellipsoidal Subspace Support Vector Data Description (ESSVDD) was proposed in [21]. ESSVDD considers the covariance of the data in the subspace and the optimization problem is given as…”

“…The criteria minimized in the previously proposed SSVDD [27], [31] and ESSVDD [21] are special cases of the proposed framework and correspond to variants GESSVDD-0-GR-min and GESSVDD-I-GR-min. We conclude that SSVDD minimizes the weighted variance of the support vectors and outliers, while having an orthogonality constraint.…”

“…Numerous extensions of OCSVM and SVDD have been proposed in the literature [25], [26]. Traditionally, the SV-based one-class classification is tackled by modeling data in the initially given feature space, but we have recently proposed one-class classification algorithms operating in an optimized lower-dimensional subspace [21], [27].…”

“…We can also formulate a spectral regression-based solution as explained in Section II-D. While earlier subspace SVDD variants [21], [27], [31] have only used gradient-based solution, we now have three alternatives: 1) gradient-based, 2) spectral, and 3) spectral regressionbased updates. Furthermore, we can pick any desired graph as L x and we note that it can be meaningful to also maximize (27) (see further discussion in Section III-B).…”