Joint rescaled asymmetric least squared nonparallel support vector machine with a stochastic quasi-Newton based algorithm

Qi, Kai; Yang, Hu

doi:10.1007/s10489-022-03183-2

Cited by 8 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Terefore, the efect of zero mean feature noise is weakened, and the fnal separating hyperplane of the CaENSVM is stable for resampling. Along with τ decreasing (close to 0), by (38), the fnal separating hyperplane is gradually dominated by the instances in S w 4 . As a result, the classifcation results are signifcantly disturbed by the zero mean feature noise around the decision boundary.…”

Section: Resampling Stability To Feature Noisementioning

confidence: 98%

“…With parameters τ and θ properly selected, equation (40) indicates that τ controls the sensitivity of the CaENSVM to feature noise. In fact, by (38), ((1 − θ) − θ(1 − y i w T x i )) and (θ(1 − y i w T x i ) + (1 − θ)) are both positive, which means that a large τ (close to 1) can well balance the size of S w 2 and S w 4 for zero mean feature noise. Terefore, the efect of zero mean feature noise is weakened, and the fnal separating hyperplane of the CaENSVM is stable for resampling.…”

Section: Resampling Stability To Feature Noisementioning

confidence: 99%

“…Te generalized quantile loss inherits the stability for resampling from the pinball loss and the robustness to outliers from Closs. Similarly, we recently proposed a joint rescaled asymmetric least squared (RaLS) loss and applied it to a nonparallel support vector machine [38]. Te proposed RaLS loss is smooth everywhere and enjoys both stability and robustness.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Capped Asymmetric Elastic Net Support Vector Machine for Robust Binary Classification

Yang

2023

International Journal of Intelligent Systems

Self Cite

View full text Add to dashboard Cite

Recently, there are lots of literature on improving the robustness of SVM by constructing nonconvex functions, but they seldom theoretically study the robust property of the constructed functions. In this paper, based on our recent work, we present a novel capped asymmetric elastic net (CaEN) loss and equip it with the SVM as CaENSVM. We derive the influence function of the estimators of the CaENSVM to theoretically explain the robustness of the proposed method. Our results can be easily extended to other similar nonconvex loss functions. We further show that the influence function of the CaENSVM is bounded, so that the robustness of the CaENSVM can be theoretically explained. Other theoretical analysis demonstrates that the CaENSVM satisfies the Bayes rule and the corresponding generalization error bound based on Rademacher complexity guarantees its good generalization capability. Since CaEN loss is concave, we implement an efficient DC procedure based on the stochastic gradient descent algorithm (Pegasos) to solve the optimization problem. A host of experiments are conducted to verify the effectiveness of our proposed CaENSVM model.

show abstract

Section: Resampling Stability To Feature Noisementioning

confidence: 98%

Section: Resampling Stability To Feature Noisementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Capped Asymmetric Elastic Net Support Vector Machine for Robust Binary Classification

Yang

2023

International Journal of Intelligent Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, ℓ 0norm is non-convex, solving the optimization with ℓ 0 -norm is an NP-hard problem. Therefore, some approximate computational methods [11][12][13][14] have been proposed. These methods integrate approximate ℓ 0 -norm function into the classifier to find the appropriate feature subset and get a better prediction accuracy.…”

Section: Introductionmentioning

confidence: 99%

Sparse accelerated limited memory BFGS algorithm for feature selection and classification

Chen

Zhang

Liu

et al. 2023

Preprint

View full text Add to dashboard Cite

Training accurate and explainable models in limited time has been a longstanding challenge for classification problems. One of the most popular techniques to cope with this challenge is support vector machine (SVM). Compared with SVM, least squares support vector machine (LSSVM) is smooth and more robust. However, LSSVM suffers from poor predictive performance and high time complexity when training large-scale and high-dimensional datasets. In this paper, we introduce a sparse accelerated limited memory Broyden–Fletcher–Goldfarb–Shanno (SAL-BFGS) algorithm which tackles this problem in two aspects: (1) We reformulate the LSSVM as a new classifier model with objective function with respect to $\ell_{0}$-norm function. (2) We propose a Nesterov's accelerated limited memory BFGS to train the classifier. Both improvements can remarkably enhance predictive accuracy and reduce computational cost of the classifier. Compared with L2SAAG, L1VRSGD, SL-BFGS, SAL-BFGS improves meidian values of TCA by an average of 6.32\%, 5.63\%, 5.44\% on twelve real-world datasets. Experimental results demonstrate the superiority of the proposed algorithm.

show abstract

“…The hinge loss is sensitive to outliers, while pinball loss is not robust enough to outliers due to its unboundedness. Since any convex loss is unbounded, many scholars have begun to focus on non-convex loss functions [11,12,20] and other types of loss functions [21][22][23], transforming the loss function to make it bounded. Scholars have conducted a lot of research on loss functions [22,24].…”

Section: Introductionmentioning

confidence: 99%

Twin Bounded Support Vector Machine with Capped Pinball Loss

Wang,

Hong,

Zhang

2023

Preprint

View full text Add to dashboard Cite

In this paper, we propose a novel classifier termed as twin bounded support vector machine with capped pinball loss (CPin-TBSVM), which has the excellent properties of being insensitive to noise. Given that the proposed model is non-convex, we use the convex-concave procedure algorithm (CCCP) to solve a series of two smaller-sized quadratic programming problems to find the optimal solution. In the process of solving the iterative sub-problem, the dual coordinate descent method (DCDM) is used for speeding up solving optimization problems. Moreover, we analyze its theoretical properties, including that the capped pinball loss satisfies Bayes’ rule and CPin-TBSVM has certain noise insensitivity and sparsity. The properties are verified on an artificial dataset as well. The numerical experiment is conducted on 24 UCI datasets and the results are compared with four other models which include SVM, TSVM, Pin-GTSVM and TPin-TSVM. The results show that the proposed CPin-TBSVM has a better classification effect and noise insensitivity.

show abstract

Joint rescaled asymmetric least squared nonparallel support vector machine with a stochastic quasi-Newton based algorithm

Cited by 8 publications

References 43 publications

Capped Asymmetric Elastic Net Support Vector Machine for Robust Binary Classification

Capped Asymmetric Elastic Net Support Vector Machine for Robust Binary Classification

Sparse accelerated limited memory BFGS algorithm for feature selection and classification

Twin Bounded Support Vector Machine with Capped Pinball Loss

Contact Info

Product

Resources

About