Towards a Unified Quadrature Framework for Large-Scale Kernel Machines

Abstract

In this paper, we develop a quadrature framework for large-scale kernel machines via a numerical multiple integration representation. Leveraging the fact that the integration domain and measure of typical kernels, e.g., Gaussian kernels, arc-cosine kernels, are fully symmetric, we introduce a deterministic fully symmetric interpolatory rule to efficiently compute its quadrature nodes and associated weights to approximate such typical kernels. This interpolatory rule is able to reduce the number of needed nodes while retaining a high approximation accuracy. Further, we randomize the above deterministic rule such that the proposed stochastic version can generate dimension-adaptive feature mappings for kernel approximation. Our stochastic rule has the nice statistical properties of unbiasedness and variance reduction with fast convergence rate. In addition, we elucidate the relationship between our deterministic/stochastic interpolatory rules and current quadrature rules for kernel approximation, including the sparse grids quadrature and stochastic spherical-radial rule, thereby unifying these methods under our framework. Experimental results on several benchmark datasets show that our fully symmetric interpolatory rule compares favorably with other representative random features based methods

Unlocking the Potential of Non-PSD Kernel Matrices: A Polar Decomposition-based Transformation for Improved Prediction Models

“…Kernel classifiers, such as Support Vector Machines (SVM) or Kernel Fisher Discriminant Analysis (KFDA), have proven to be powerful tools in a variety of machine learning problems [21,25,36,47,49]. In general, the performance of a kernel classifier heavily relies on the choice of the respective kernel function 𝑘 (•, •) and its ability to effectively capture the data's underlying structure, such as similarity or dissimilarity between data points.…”