A geometry‐based two‐step method for nonlinear classification using quasi‐linear support vector machine

Abstract

This paper proposes a two‐step method to construct a nonlinear classifier consisting of multiple local linear classifiers interpolated with a basis function. In the first step, a geometry‐based approach is first introduced to detect local linear partitions and build local linear classifiers. A coarse nonlinear classifier can then be constructed by interpolating the local linear classifiers. In the second step, a support vector machine (SVM) formulation is used to further implicitly optimize the linear parameters of the nonlinear classifier. In this way, the nonlinear classifier is constructed in exactly the same way as a standard SVM, using a special data‐dependent quasi‐linear kernel composed of the information of the local linear partitions. Numerical experiments on several real‐world datasets demonstrate the effectiveness of the proposed classifier and show that, in cases where traditional nonlinear SVMs run into overfitting problems, the proposed classifier is effective in improving the classification performance. © 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Quasi‐linear SVM classifier with segmented local offsets for imbalanced data classification

“…Before applying the SVM optimization, we need to determine the parameters μ j and σ j . For the purpose, we first introduce a modification to the geometry‐based partitioning algorithm for the imbalanced dataset and apply it to divide the input space of training data into M linearly separable local partitions along the separation boundary.…”

“…The geometry‐based algorithm for detecting the local linear partitions can be found in Refs . The modified version for imbalanced dataset is briefly described by the following.…”

“…Aiming to solve the above problems, in this paper, we propose a simple yet effective SVM classifier with local offset adjustment for imbalanced classification problems. (i) To deal with different imbalance distribution within datasets, we first introduce a modification to the geometry‐based partitioning method for imbalanced datasets and apply it to divide the imbalanced datasets into multiple local linearly separable partitions along the potential separation boundary . (ii) To obtain local linear classifiers in each partition, an SVM method is used to construct a quasi‐linear SVM classifier which is a global optimization of local linear classifiers by composing a quasi‐linear kernel with the information of local linear partitions .…”

An autoencoder‐based piecewise linear model for nonlinear classification using quasilinear support vector machines

“…Another algorithmic solution is a combination of several local linear models because it is more efficient to train linear models. Inspired by the theory of locally linear approximation of nonlinear functions, many algorithms solve nonlinear classification problems by composing a set of local linear classifiers . They assume that the data instances in a small local region are more linearly separable.…”

Multiple kernel learning using nonlinear lasso

“…Kernel methods such as support vector machines (SVMs) and kernel Fisher discriminant analysis (KFDA) have been successfully applied to a wide variety of machine learning problems . These methods map data points from the input space to some feature space, that is, a higher dimensional reproducing kernel Hilbert space (RKHS), where even relatively simple algorithms such as linear methods can deliver very impressive performance.…”