Multiple Kernel SVM Based on Two-Stage Learning

Abstract

In this paper we introduce the idea of two-stage learning for multiple kernel SVM (MKSVM) and present a new MKSVM algorithm based on two-stage learning (MKSVM-TSL). The first stage is the pre-learning and its aim is to obtain the information of data such that the "important" samples for classification can be generated in the formal learning stage and these samples are uniformly ergodic Markov chain (u.e.M.c.). To study comprehensively the proposed MKSVM-TSL algorithm, we estimate the generalization bound of MKSVM based on u.e.M.c. samples and obtain its fast learning rate. And in order to show the performance of the proposed MKSVM-TSL algorithm for better, we also perform the numerical experiments on various publicly available datasets. From the experimental results, we can find that compared to three classical multiple kernel learning (MKL) algorithms, the proposed MKSVM-TSL algorithm has better performance in three aspects of the total time of sampling and training, the accuracy and the sparsity of classifiers, respectively. INDEX TERMS Two-stage learning, Multiple kernel SVM, Uniformly ergodic Markov chain, Learning rate.

Generative-discriminative machine learning models for high-frequency financial regime classification

“…The reasons for focusing on multiple kernel learning (MKL) methods are threefold: the MKL approach is essentially a dynamic ensemble method that constructs a mixture kernel, thereby encoding complementary information. Secondly, it is known that MKL methods are appropriate for small but wide datasets [40,41], as is the case with our high-dimensional generative embeddings extracted from the HMM. The MKL approach involves integration of different kernels, such as polynomials and RBF helps MKL to capture nonlinear decision boundaries even with limited data, and the kernel-based regularization makes it less likely it will overfit [42].…”