Robust linear and support vector regression

Abstract

AbstractÐThe robust Huber M-estimator, a differentiable cost function that is quadratic for small errors and linear otherwise, is modeled exactly, in the original primal space of the problem, by an easily solvable simple convex quadratic program for both linear and nonlinear support vector estimators. Previous models were significantly more complex or formulated in the dual space and most involved specialized numerical algorithms for solving the robust Huber linear estimator [3] [28]. Numerical test comparisons with these algorithms indicate the computational effectiveness of the new quadratic programming model for both linear and nonlinear support vector problems. Results are shown on problems with as many as 20,000 data points, with considerably faster running times on larger problems.

Untitled

“…The original algorithms for the pattern recognition and the regression SVM appeared in Mangasarian and Musicant (1999b) and Mangasarian and Musicant (1999a) respectively. This method redefines the formulation of the SVM problem to obtain the quadratic program lacking the equality constraint.…”

Robust convex quadratically constrained programs

“…Since the uncertainty sets S m × S v × S f and S d are special cases of the factorized uncertainty structure proposed in (19), (39) can be reduced to an SOCP. For details on robust portfolio selection problems and the performance on real market data see [16].…”

“…Lemma 3 below establishes that a robust quadratic constraint corresponding to (19) can be reformulated as a collection of linear and SOC constraints.…”

“…The decision vector x ∈ R n satisfies the robust convex quadratic constraint x T Qx + 2q T x + γ ≤ 0 for all (Q, q, γ) ∈ S e , where S e is defined by (19), if and only if there exist τ, ν, σ, r ∈ R, u ∈ R n and w ∈ R m such that…”

“…In a typical application of linear discrimination, the hyperplane (w, b) is chosen by solving the following quadratic program [8,19,28].…”