Proximal Operator and Optimality Conditions for Ramp Loss SVM

Abstract

Support vector machines with ramp loss (dubbed as L r -SVM) have attracted wide attention due to the boundedness of ramp loss. However, the corresponding optimization problem is non-convex and the given Karush-Kuhn-Tucker (KKT) conditions are only the necessary conditions. To enrich the optimality theory of L r -SVM and go deep into its statistical nature, we first introduce and analyze the proximal operator for ramp loss, and then establish a stronger optimality conditions: P-stationarity, which is proved to be the first-order necessary and sufficient conditions for local minimizer of L r -SVM. Finally, we define the L r support vectors based on the concept of P-stationary point, and show that all L r support vectors fall into the support hyperplanes, which possesses the same feature as the one of hard margin SVM.