Support Vector Machines with Applications

Abstract

Support vector machines (SVMs) appeared in the early nineties as optimal margin classifiers in the context of Vapnik's statistical learning theory. Since then SVMs have been successfully applied to real-world data analysis problems, often providing improved results compared with other techniques. The SVMs operate within the framework of regularization theory by minimizing an empirical risk in a well-posed and consistent way. A clear advantage of the support vector approach is that sparse solutions to classification and regression problems are usually obtained: only a few samples are involved in the determination of the classification or regression functions. This fact facilitates the application of SVMs to problems that involve a large amount of data, such as text processing and bioinformatics tasks. This paper is intended as an introduction to SVMs and their applications, emphasizing their key features. In addition, some algorithmic extensions and illustrative real-world applications of SVMs are shown.Comment: This paper commented in: [math/0612820], [math/0612821], [math/0612822], [math/0612824]. Rejoinder in [math.ST/0612825]. Published at http://dx.doi.org/10.1214/088342306000000493 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Graduating the age-specific fertility pattern using Support Vector Machines

“…In particular, the values ε = 0.0001, σ = 40 and C = 1.8, have been chosen for the SVM implementation. Additionally, in this application, the values for the corresponding dimensions in the SVM model are n = 1, m = ∞ (given that this is the dimension induced by the Gaussian kernel, see Moguerza and Muñoz 2006) and p = 34, that is, the number of data within each set. We should notice here again that we use the same set of parameter values for all the data sets.…”

“…It can be shown (Moguerza and Muñoz 2006) that the regularization problem can be formulated as a convex quadractic optimization problem (therefore, without local minima) of the form:…”

“…The Gaussian kernel, given its approximation capacity, is the most extensively used (see Moguerza and Muñoz 2006, for a complete set of examples). In practice, the optimization problem to solve is not the primal formulation shown above.…”

“…Support Vector Machines (SVM) is a modern non parametric graduation technique that appeared in the mid nineties in the framework of Vapnik's Statistical Learning Theory (Vapnik 1995;Moguerza and Muñoz 2006). Since SVM techniques have shown very successful results in smoothing noisy data, such as neighbourhood curves (Muñoz and Moguerza 2005) or nonlinear profiles (Moguerza, Muñoz, and Psarakis 2007), they can probably serve as useful tools for fertility graduation purposes too.…”

Supervised classification and mathematical optimization

“…This geometrical optimization problem can be written as a convex quadratic optimization problem with linear constraints, in principle solvable by any nonlinear optimization procedure. See also [18,40,108,177,113] for introductory surveys on SVMs.…”