An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

Abstract

Support vector machines (SVMs) with sparsityinducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconvexity. In this paper, we propose an efficient ADMM-based algorithm to the nonconvex penalized SVMs. The proposed algorithm covers a large class of commonly used nonconvex regularization terms including the smooth clipped absolute deviation (SCAD) penalty, minimax concave penalty (MCP), log-sum penalty (LSP) and capped-ℓ1 penalty. The computational complexity analysis shows that the proposed algorithm enjoys low computational cost. Moreover, the convergence of the proposed algorithm is guaranteed. Extensive experimental evaluations on five benchmark datasets demonstrate the superior performance of the proposed algorithm to other three state-of-the-art approaches.

A Parallel Algorithm for Large-Scale Nonconvex Penalized Quantile Regression

“…Now, we will derive convergence conclusions lim k→+∞ w k − w k+1 2 = 0, which is similar to Guan et al (2018).…”

“…Besides, we also consider the new nonconvex penalty, named Cnet, which is the combination of capped-ℓ 1 and ridge. The capped-ℓ 1 is proposed by and has received some attention in Guan et al (2018) and Pan et al (2021), which is defined as λ 1 j min{|β j |, a}. Then, the Cnet is defined as…”

“…Similar to the steps of QPADM before the iterative process, the only difference is that the r subproblem is replaced by the u and v subproblems. In addition, compared to the approximate solution of β in the QPADM, QPADMslack uses the approximate closed-form solution for SCAD and MCP proposed by Gong et al (2013) and Guan et al (2018). The so-called approximate closed-form solution means that it obtains the minimum point by comparing the minimum value of the objective function among several candidate minimum points.…”