Is Primal Better Than Dual

Abstract

Chapelle proposed to train support vector machines (SVMs) in the primal form by Newton's method and discussed the advantages of training SVMs in the primal. In this paper we propose training L2 SVMs in the dual form in the similar way that Chapelle proposed. Namely, we solve the quadratic programming problem for the initial working set of training data by Newton's method, delete from the working set the data with negative Lagrange multipliers as well as the data with the associated margins larger than or equal to 1, add to the working set training data with the associated margins less than 1, and repeat training the SVM until the working set does not change. The matrix associated with the dual quadratic form is positive definite while that with the primal quadratic form is positive semi-definite. And the former matrix requires less kernel evaluation. Computer experiments show that for most cases training the SVM by the proposed method is more stable and faster than training the SVM in the primal.

Convergence improvement of active set support vector training

“…Thus, convergence of the above procedure is not guaranteed. But in most cases, training converges [10].…”

“…In the following we summarize active set training dis cussed in [10]. Assume that we can estimate the index set of support vectors 8 correctly.…”

“…In [10], this method is extended to dual form. Namely, the L2 SVM is trained by Newton's method using the data in the working set selected from the training data.…”

“…In section II we describe L2 SVMs and in section III, we summarize ac tive set training discussed in [10]. Then in section IV, we discuss the proposed method.…”