Evolutionary tuning of multiple SVM parameters

Friedrichs, Frauke; Igel, Christian

doi:10.1016/j.neucom.2004.11.022

Cited by 448 publications

(218 citation statements)

References 13 publications

Supporting

Mentioning

208

Contrasting

Order By: Relevance

“…Multi-scale kernels have been mainly used with evolutionary algorithms [19,32,17] or gradient-based methods for specific applications [36,6,24]. The main problem with evolutionary approaches is the high computational cost and the necessity of tuning a large number of parameters associated to the algorithm.…”

Section: Multi-scale Casementioning

confidence: 99%

“…The general motivation for the use of multi-scale kernels is that, in real-world applications, the attributes can present very different nature, which hampers the performance of spherical kernels (i.e., with the same kernel width for each attribute) [24,17]. However, the number of parameters (as many as the number of features) makes the computational cost prohibitive when considering a cross-validation technique.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

Pérez-Ortiz

Gutiérrez

Sánchez‐Monedero

et al. 2015

Neural Process Lett

View full text Add to dashboard Cite

Section: Multi-scale Casementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

Pérez-Ortiz

Gutiérrez

Sánchez‐Monedero

et al. 2015

Neural Process Lett

View full text Add to dashboard Cite

“…The Covariance Matrix Adaptation Evolution Strategy (Hansen and Ostermeier 2001) is a modification of standard genetic algorithm that updates the covariance matrix used for selecting the next generation. Friedrichs and Igel (2005) used this method to optimize multiple parameters of Support Vector Machines (SVM). Merelo et al (2001) presented G-LVQ, which uses a genetic algorithm to optimize Learning Vector Quantization (Kohonen 1998).…”

Section: Introductionmentioning

confidence: 99%

Meta-learning for evolutionary parameter optimization of classifiers

2012

View full text Add to dashboard Cite

The performance of most of the classification algorithms on a particular dataset is highly dependent on the learning parameters used for training them. Different approaches like grid search or genetic algorithms are frequently employed to find suitable parameter values for a given dataset. Grid search has the advantage of finding more accurate solutions in general at the cost of higher computation time. Genetic algorithms, on the other hand, are able to find good solutions in less time, but the accuracy of these solutions is usually lower than those of grid search. This paper uses ideas from meta-learning and case-based reasoning to provide good starting points to the genetic algorithm. The presented approach reaches the accuracy of grid search at a significantly lower computational cost. We performed extensive experiments for optimizing learning parameters of the Support Vector Machine (SVM) and the Random Forest classifiers on over 100 datasets from UCI and StatLib repositories. For the SVM classifier, grid search achieved an average accuracy of 81 % and took six hours for training, whereas the standard genetic algorithm obtained 74 % accuracy in close to one hour of training. Our method was able to achieve an average accuracy of 81 % in only about 45 minutes. Similar results were achieved for the Random Forest classifier. Besides a standard genetic algorithm, we also compared the presented method with three state-of-the-art optimization algorithms: Generating Set Search, Dividing Rectangles, and the Covariance Matrix Adaptation Evolution Strategy. Experimental results show that our method achieved the highest average accuracy for both classifiers. Our approach can be particularly useful when training classifiers on large datasets where grid search is not feasible.

show abstract

“…Several methods for model selection in SVM and SVM-like models were proposed, e.g. in [9,10,6,7,11,8]. The popular way is application of Bayesian learning framework and maximal evidence principle [3].…”

Section: Introductionmentioning

confidence: 99%

The Use of Stability Principle for Kernel Determination in Relevance Vector Machines

Kropotov

Vetrov

Ptashko

et al. 2006

Neural Information Processing

View full text Add to dashboard Cite

Abstract. The task of RBF kernel selection in Relevance Vector Machines (RVM) is considered. RVM exploits a probabilistic Bayesian learning framework offering number of advantages to state-of-the-art Support Vector Machines. In particular RVM effectively avoids determination of regularization coefficient C via evidence maximization. In the paper we show that RBF kernel selection in Bayesian framework requires extension of algorithmic model. In new model integration over posterior probability becomes intractable. Therefore point estimation of posterior probability is used. In RVM evidence value is calculated via Laplace approximation. However, extended model doesn't allow maximization of posterior probability as dimension of optimization parameters space becomes too high. Hence Laplace approximation can be no more used in new model. We propose a local evidence estimation method which establishes a compromise between accuracy and stability of algorithm. In the paper we first briefly describe maximal evidence principle, present model of kernel algorithms as well as our approximations for evidence estimation, and then give results of experimental evaluation. Both classification and regression cases are considered.

show abstract

Evolutionary tuning of multiple SVM parameters

Cited by 448 publications

References 13 publications

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

A Study on Multi-Scale Kernel Optimisation via Centered Kernel-Target Alignment

Meta-learning for evolutionary parameter optimization of classifiers

The Use of Stability Principle for Kernel Determination in Relevance Vector Machines

Contact Info

Product

Resources

About