CROification: Accurate Kernel Classification with the Efficiency of Sparse Linear SVM

Kafai, Mehran; Eshghi, Kave

doi:10.1109/tpami.2017.2785313

Cited by 45 publications

(26 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…• RF [12]: It is a nonparametric kernel learning framework by learning from optimal random features. • CROiclassification [57]: A new CRO (Concomitant Rank Order) kernel is proposed to approximate the Gaussian kernel on the unit sphere by random features. The used kernel in CROiclassification [57] is a Gaussian kernel.…”

Section: Compared With Other Kernel Approximation Methods With Bocmentioning

confidence: 99%

“…• CROiclassification [57]: A new CRO (Concomitant Rank Order) kernel is proposed to approximate the Gaussian kernel on the unit sphere by random features. The used kernel in CROiclassification [57] is a Gaussian kernel. Instead, as a data-dependent method, RF considers the learned random features for kernel learning and approximation.…”

Section: Compared With Other Kernel Approximation Methods With Bocmentioning

confidence: 99%

“…V. We find that RF appears to obtain a not very promising performance even if the kernel is learned instead of directly specified. Compared to [57] equipped with the Gaussian kernel, our method with the polynomial kernel achieves a comparable classification performance and computational cost. Actually, in this paper we do not want to claim that our RFF-DIGMM model is better than other kernel approximation methods, as the scope of their applications are not the same.…”

Section: Compared With Other Kernel Approximation Methods With Bocmentioning

confidence: 99%

See 2 more Smart Citations

A Double-Variational Bayesian Framework in Random Fourier Features for Indefinite Kernels

Liu

Huang

Shi

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Random Fourier features (RFF) have been successfully employed to kernel approximation in large scale situations. The rationale behind RFF relies on Bochner's theorem, but the condition is too strict and excludes many widely-used kernels, e.g., dot-product kernels and indefinite kernels. In this paper, we present a unified RFF framework for indefinite kernel approximation in the Reproducing Kernel Kreȋn Spaces (RKKS). Besides, our model is also suited to approximate a dot-product kernel on the unit sphere, as it can be transformed into a shiftinvariant but indefinite kernel. By the Kolmogorov decomposition scheme, an indefinite kernel in RKKS can be decomposed into the difference of two unknown positive definite (PD) kernels. The spectral distribution of each underlying PD kernel can be formulated as a nonparametric Bayesian Gaussian mixtures model. Based on this, we propose a double-infinite Gaussian mixture model in RFF by placing the Dirichlet process prior. It takes full advantage of high flexibility on the number of components and has the capability of approximating indefinite kernels on a wide scale. In model inference, we develop a non-conjugate variational algorithm with a sub-sampling scheme for posterior inference. It allows for the non-conjugate case in our model and is quite efficient due to the sub-sampling strategy. Experimental results on several large classification datasets demonstrate the effectiveness of our nonparametric Bayesian model for indefinite kernel approximation when compared to other representative random features based methods.

show abstract

Section: Compared With Other Kernel Approximation Methods With Bocmentioning

confidence: 99%

Section: Compared With Other Kernel Approximation Methods With Bocmentioning

confidence: 99%

Section: Compared With Other Kernel Approximation Methods With Bocmentioning

confidence: 99%

See 1 more Smart Citation

A Double-Variational Bayesian Framework in Random Fourier Features for Indefinite Kernels

Liu

Huang

Shi

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

show abstract

“…The general fitting algorithm has difficulty performing multidimensional data fitting. Support vector machines (SVMs) have significant advantages in solving high-dimensional, nonlinear, and other small-sample identification problems or fitting problems [34]- [45]. The purpose of this paper is to construct a curvature demodulation method that adopts complete spectral information.…”

Section: The Variety Of Surrounding Environmental Physical Quantitiesmentioning

confidence: 99%

Mach-Zehnder Interferometer Sensor Curvature Demodulation Method Based on the Orthogonal Decomposition of Spectral Curves

et al. 2020

View full text Add to dashboard Cite

This paper proposes a curvature demodulation of Mach-Zehnder interferometer (MZI) sensors based on complete spectral information. An MZI sensor composed of two waist-enlarged single mode fibers (SMFs) was fabricated. To research the spectral characteristics of the sensor under different curvatures, an experimental system was built to load curvature and save spectra. By analyzing the influence of light intensity and wavelength on the curvature fitting accuracy, researchers have found that complete spectral information can express curvature accurately. This paper introduced the concept of orthogonal decomposition to extract complete spectral information. A group of Chebyshev polynomials was chosen as the basis to decompose the spectral curve into a vector (spectral curve decomposition coefficients). A support vector machine improved by a genetic algorithm (GA-SVM) was used to fit the relationship between the spectral curve decomposition coefficients and the curvature. This paper introduces a new accuracy estimation parameter (''equivalent absolute error''), which is used to measure the accuracy of demodulation. The equivalent absolute error of the method proposed in this paper is 0.0022494 m −1 , which is superior to other methods. The advantage of this demodulation method is that the curvature is demodulated independently of the spectral information about a particular peak (valley). The demodulation method is applicable to a demodulation process of multiple physical quantities based on spectral information.

show abstract

“…In the past few decades, kernel method, as an excellent and powerful theory tool, has intensely received attention in many research fields, e.g. support vector machine (SVM) [1 ] and transfer regression (TR) [2 ]. For solving the non‐linear filtering problems of input space, a kernel adaptive filter (KAF) [3 ], transforming the input data into the reproducing kernel Hilbert space, has been widely studied and rapidly developed in the adaptive signal processing area.…”

Section: Introductionmentioning

confidence: 99%

Quantised kernel least mean square algorithm with a learning vector strategy

Zhang

Wang

2020

Electron. lett.

View full text Add to dashboard Cite

To decrease the network size of quantised kernel least mean square (QKLMS) dramatically, the QKLMS algorithm with an online learning vector strategy, which is named LV‐QKLMS, is proposed in this Letter. The centres of the dictionary in LV‐QKLMS are updated dynamically by the online learning vectors. Unlike QKLMS only updating the coefficient of the nearest centre to current input, LV‐QKLMS updates the centres of the dictionary using the redundant input data accordingly. Thanks to the iterative learning vector, LV‐QKLMS has a constantly changing quantisation area on the arrival of a new sample, and thus generates a superimposed quantisation area containing more redundant input data than QKLMS with a fixed quantisation area. In addition, the learning vector updated by a linear combination of input data and itself in the superimposed quantisation area guarantees the filtering accuracy of LV‐QKLMS. Simulation results on the prediction of Mackey–Glass chaotic time series are presented to validate the effectiveness of LV‐QKLMS.

show abstract

CROification: Accurate Kernel Classification with the Efficiency of Sparse Linear SVM

Cited by 45 publications

References 23 publications

A Double-Variational Bayesian Framework in Random Fourier Features for Indefinite Kernels

A Double-Variational Bayesian Framework in Random Fourier Features for Indefinite Kernels

Mach-Zehnder Interferometer Sensor Curvature Demodulation Method Based on the Orthogonal Decomposition of Spectral Curves

Quantised kernel least mean square algorithm with a learning vector strategy

Contact Info

Product

Resources

About