The Use of Stability Principle for Kernel Determination in Relevance Vector Machines

Abstract

Abstract. The task of RBF kernel selection in Relevance Vector Machines (RVM) is considered. RVM exploits a probabilistic Bayesian learning framework offering number of advantages to state-of-the-art Support Vector Machines. In particular RVM effectively avoids determination of regularization coefficient C via evidence maximization. In the paper we show that RBF kernel selection in Bayesian framework requires extension of algorithmic model. In new model integration over posterior probability becomes intractable. Therefore point estimation of posterior probability is used. In RVM evidence value is calculated via Laplace approximation. However, extended model doesn't allow maximization of posterior probability as dimension of optimization parameters space becomes too high. Hence Laplace approximation can be no more used in new model. We propose a local evidence estimation method which establishes a compromise between accuracy and stability of algorithm. In the paper we first briefly describe maximal evidence principle, present model of kernel algorithms as well as our approximations for evidence estimation, and then give results of experimental evaluation. Both classification and regression cases are considered.

Research on Prediction Method of Alloy Element Yield in Smelting Stage of Iron and Steel Product Based on Improved Support Vector Regression

“…4.1 Parameter optimization method for SVR based on ant colony algorithm The performance of SVR depends largely on the selection of parameters. RBF kernel function with fewer parameters has advantage of the nonlinear problem [6] , so parameters needed to optimize include penalty parameter C , nuclear parameter γ and sensitive coefficient ε .Ant colony algorithm with quick calculation speed, which is not affected by the complexity of the actual problem, is used to realize parameter optimization in this paper.…”