L<inf>p</inf>-regularized optimization by using orthant-wise approach for inducing sparsity

Abstract

Sparsity induced in the optimized weights effectively works for factorization with robustness to noises and for classification with feature selection. For enhancing the sparsity, L 1 regularization is introduced into the objective cost function to be minimized. In general, however, L p (p < 1) regularization leads to more sparse solutions than L 1 , though L p regularized problem is difficult to be effectively optimized. In this paper, we propose a method to efficiently optimize the L p regularized problem. The method reduces the L p problem into L 1 regularized one via transforming target variables by the mapping based on L p , and optimizes it by using orthant-wise approach. In the proposed method, the L p problem is directly optimized for computational efficiency without reformulating it into iteratively reweighting scheme. The proposed method is generally applicable to various problems with L p regularization, such as factorization and classification. In the experiments on the classification using logistic regression and factorization based on least squares, the proposed method produces favorable sparse results.