Nonconvex Regularizations for Feature Selection in Ranking With Sparse SVM

Laporte, Léa; Flamary, Rémi; Canu, Stéphane; Déjean, Sébastien; Mothe, Josiane

doi:10.1109/tnnls.2013.2286696

Cited by 94 publications

(68 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Zhang et al [2] established a unified theory for SCAD-and MCP-penalized SVM in the high-dimensional setting. Laporte et al [16] proposed a general framework for feature selection in learning to rank using SVM with nonconvex regularizations such as log penalty, MCP and ℓ p norm with 0 < p < 1. Recently, Zhang et al [17] have established an unified theory for a general class of nonconvex penalized SVMs in the high-dimensional setting.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

Guan

Qiao

et al. 2018

2018 IEEE International Conference on Data Mining Workshops (ICDMW)

View full text Add to dashboard Cite

Support vector machines (SVMs) with sparsityinducing nonconvex penalties have received considerable attentions for the characteristics of automatic classification and variable selection. However, it is quite challenging to solve the nonconvex penalized SVMs due to their nondifferentiability, nonsmoothness and nonconvexity. In this paper, we propose an efficient ADMM-based algorithm to the nonconvex penalized SVMs. The proposed algorithm covers a large class of commonly used nonconvex regularization terms including the smooth clipped absolute deviation (SCAD) penalty, minimax concave penalty (MCP), log-sum penalty (LSP) and capped-ℓ1 penalty. The computational complexity analysis shows that the proposed algorithm enjoys low computational cost. Moreover, the convergence of the proposed algorithm is guaranteed. Extensive experimental evaluations on five benchmark datasets demonstrate the superior performance of the proposed algorithm to other three state-of-the-art approaches.

show abstract

Section: Related Workmentioning

confidence: 99%

“…• the successive quadratic algorithm for the SCADpenalized hinge loss function (SCAD SVM 2 ) [11]. • the reweighted ℓ 1 scheme for the MCP-penalized squared hinge loss function (RankSVM-MCP 3 ) [16]. • the generative shrinkage and thresholding (GIST 4 ) algorithm [12].…”

Section: Experimental Evaluation a Experimental Setupmentioning

confidence: 99%

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

Guan

Qiao

et al. 2018

2018 IEEE International Conference on Data Mining Workshops (ICDMW)

View full text Add to dashboard Cite

show abstract

“…11] for p with p = 1/2). Note that efficient methods which estimate the Hessian matrix [61], [62] exist, as well as a wide range of methods based on DC programming, which have shown to work very well in practice [62], [63] and can handle the general case p ∈ (0, 1] for the p pseudonorm (see [64] for an implementation).…”

Section: E Optimization Algorithmsmentioning

confidence: 99%

Nonconvex Regularization in Remote Sensing

Tuia

Flamary

Barlaud

2016

IEEE Trans. Geosci. Remote Sensing

Self Cite

View full text Add to dashboard Cite

In this paper, we study the effect of different regularizers and their implications in highdimensional image classification and sparse linear unmixing. Although kernelization or sparse methods are globally accepted solutions for processing data in high dimensions, we present here a study on the impact of the form of regularization used and its parameterization.We consider regularization via traditional squared (!2) and sparsity-promoting (!1) norms, as well as more unconventional non convex regularizers (!p and log sum penalty). We compare their properties and advantages on several classification and linear unmoving tasks and provide advices on the choice of the best regularizer for the problemat hand. Finally,we also provide a fully functional toolbox for the community. This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 1 Nonconvex Regularization in Remote SensingDevis Tuia, Senior Member, IEEE, Rémi Flamary, and Michel Barlaud, Senior Member, IEEE Abstract-In this paper, we study the effect of different regularizers and their implications in high-dimensional image classification and sparse linear unmixing. Although kernelization or sparse methods are globally accepted solutions for processing data in high dimensions, we present here a study on the impact of the form of regularization used and its parameterization. We consider regularization via traditional squared ( 2 ) and sparsity-promoting ( 1 ) norms, as well as more unconventional nonconvex regularizers ( p and log sum penalty). We compare their properties and advantages on several classification and linear unmixing tasks and provide advices on the choice of the best regularizer for the problem at hand. Finally, we also provide a fully functional toolbox for the community.

show abstract

“…Existing feature selection algorithm can be categorized as supervised feature selection (on data with full class labels) [5]- [9], unsupervised feature selection (on data without class labels) [10]- [15], and semisupervised feature selection (on data with partial labels) [14], [16], [17]. Feature selection in unsupervised context is considered to be more difficult than the other two cases, since there is no target information available for training.…”

Section: Introductionmentioning

confidence: 99%

Simultaneous Bayesian Clustering and Feature Selection Through Student’s ${t}$ Mixtures Model

Sun

Zhou

Keates

et al. 2018

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Abstract-In this paper, we proposed a generative model for feature selection under the unsupervised learning context. The model assumes that data are independently and identically sampled from a finite mixture of Student's t distributions, which can reduce the sensitiveness to outliers. Latent random variables that represent the features' salience are included in the model for the indication of the relevance of features. As a result, the model is expected to simultaneously realize clustering, feature selection, and outlier detection. Inference is carried out by a tree-structured variational Bayes algorithm. Full Bayesian treatment is adopted in the model to realize automatic model selection. Controlled experimental studies showed that the developed model is capable of modeling the data set with outliers accurately. Furthermore, experiment results showed that the developed algorithm compares favorably against existing unsupervised probability model-based Bayesian feature selection algorithms on artificial and real data sets. Moreover, the application of the developed algorithm on real leukemia gene expression data indicated that it is able to identify the discriminating genes successfully.

show abstract

Nonconvex Regularizations for Feature Selection in Ranking With Sparse SVM

Cited by 94 publications

References 34 publications

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

An Efficient ADMM-Based Algorithm to Nonconvex Penalized Support Vector Machines

Nonconvex Regularization in Remote Sensing

Simultaneous Bayesian Clustering and Feature Selection Through Student’s ${t}$ Mixtures Model

Contact Info

Product

Resources

About