A New Algorithm for Training SVMs Using Approximate Minimal Enclosing Balls

Frandi, Emanuele; Gasparo, M. G.; Lodi, Stefano; Ñanculef, Ricardo; Sartori, Claudio

doi:10.1007/978-3-642-16687-7_16

Cited by 5 publications

(16 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, we conclude that MFW is more accurate than FW. This last observation stresses the relevance of this work as an extension of the results presented in [10].…”

Section: Statistical Testssupporting

confidence: 83%

See 1 more Smart Citation

Training Support Vector Machines Using Frank–wolfe Optimization Methods

Frandi

Ñanculef

Gasparo

et al. 2013

Int. J. Patt. Recogn. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

Training a Support Vector Machine (SVM) requires the solution of a quadratic programming problem (QP) whose computational complexity becomes prohibitively expensive for large scale datasets. Traditional optimization methods cannot be directly applied in these cases, mainly due to memory restrictions.By adopting a slightly different objective function and under mild conditions on the kernel used within the model, efficient algorithms to train SVMs have been devised under the name of Core Vector Machines (CVMs). This framework exploits the equivalence of the resulting learning problem with the task of building a Minimal Enclosing Ball (MEB) problem in a feature space, where data is implicitly embedded by a kernel function.In this paper, we improve on the CVM approach by proposing two novel methods to build SVMs based on the Frank-Wolfe algorithm, recently revisited as a fast method to approximate the solution of a MEB problem. In contrast to CVMs, our algorithms do not require to compute the solutions of a sequence of increasingly complex QPs and are defined by using only analytic optimization steps. Experiments on a large collection of datasets show that our methods scale better than CVMs in most cases, sometimes at the price of a slightly lower accuracy. As CVMs, the proposed methods can be easily extended to machine learning problems other than binary classification. However, effective classifiers are also obtained using kernels which do not satisfy the condition required by CVMs and can thus be used for a wider set of problems.

show abstract

“…However, we conclude that MFW is more accurate than FW. This last observation stresses the relevance of this work as an extension of the results presented in [10].…”

Section: Statistical Testssupporting

confidence: 83%

“…5.3 and 5.4. In Subsection 5.5 we present additional experiments on the set of problems studied in [10]. The statistical significance of the results presented so far is analyzed in section 5.6.…”

Section: Organization Of This Sectionmentioning

confidence: 99%

Training Support Vector Machines Using Frank–wolfe Optimization Methods

Frandi

Ñanculef

Gasparo

et al. 2013

Int. J. Patt. Recogn. Artif. Intell.

Self Cite

View full text Add to dashboard Cite

show abstract

“…In addition, several variants of the basic procedure have been analyzed, which can improve the convergence rate and practical performance of the basic FW iteration [15,35,26,6]. From a practical point of view, they have emerged as efficient alternatives to traditional methods in several contexts, such as large-scale SVM classification [7,8,35,6] and nuclear norm-regularized matrix recovery [22,42]. In view of these developements, FW algorithms have come to be regarded as a suitable approach to large-scale optimization in various areas of Machine Learning, statistics, bioinformatics and related fields [1,27].…”

Section: Frank-wolfe Optimizationmentioning

confidence: 99%

“…In Algorithm 2, we compute the coordinates of the gradient using the method of residuals given by equation (7). Due to the randomization, this method becomes very advantageous with respect to the use of the alternative method based on the active covariates, even for very large p. Indeed, if we denote by s the cost of performing a dot product between a predictor z i and another vector in R m , the overall cost of picking out the FW vertex in step 1 of our algorithm is O(s|S|).…”

Section: Complexity and Implementation Detailsmentioning

confidence: 99%

Fast and scalable Lasso via stochastic Frank–Wolfe methods with a convergence guarantee

et al. 2016

Self Cite

View full text Add to dashboard Cite

Frank-Wolfe (FW) algorithms have been often proposed over the last few years as efficient solvers for a variety of optimization problems arising in the field of machine learning. The ability to work with cheap projection-free iterations and the incremental nature of the method make FW a very effective choice for many large-scale problems where computing a sparse model is desirable. In this paper, we present a high-performance implementation of the FW method tailored to solve large-scale Lasso regression problems, based on a randomized iteration, and prove that the convergence guarantees of the standard FW method are preserved in the stochastic setting. We show experimentally that our algorithm outperforms several existing state of the art methods, including the Coordinate Descent algorithm by Friedman et al. (one of the fastest known Lasso solvers), on several benchmark datasets with a very large number of features, without sacrificing the accuracy of the model. Our results illustrate that the algorithm is able to generate the complete regularization path on problems of size up to four million variables in <1 min.

show abstract

“…The solution of the linear approximation step can be obtained in a fast way by solving a largest eigenvalue problem, as opposed to proximal methods that require a full SVD of the gradient matrix at each iteration, which is prohibitive for large-scale problems. As a motivating example, we consider the SVM problem (5) for the experiments in this paper, not only because of its significance, but also to allow for a comparison with the results obtained in previous research efforts [19,20,11,7,21].…”

Section: Applications To Machine Learningmentioning

confidence: 99%

A PARTAN-accelerated Frank-Wolfe algorithm for large-scale SVM classification

Frandi

Ñanculef

Suykens

2015

2015 International Joint Conference on Neural Networks (IJCNN)

Self Cite

View full text Add to dashboard Cite

Frank-Wolfe algorithms have recently regained the attention of the Machine Learning community. Their solid theoretical properties and sparsity guarantees make them a suitable choice for a wide range of problems in this field. In addition, several variants of the basic procedure exist that improve its theoretical properties and practical performance. In this paper, we investigate the application of some of these techniques to Machine Learning, focusing in particular on a Parallel Tangent (PARTAN) variant of the FW algorithm that has not been previously suggested or studied for this type of problems. We provide experiments both in a standard setting and using a stochastic speed-up technique, showing that the considered algorithms obtain promising results on several medium and large-scale benchmark datasets for SVM classification.

show abstract

A New Algorithm for Training SVMs Using Approximate Minimal Enclosing Balls

Cited by 5 publications

References 8 publications

Training Support Vector Machines Using Frank–wolfe Optimization Methods

Training Support Vector Machines Using Frank–wolfe Optimization Methods

Fast and scalable Lasso via stochastic Frank–Wolfe methods with a convergence guarantee

A PARTAN-accelerated Frank-Wolfe algorithm for large-scale SVM classification

Contact Info

Product

Resources

About