Large-scale sparse logistic regression

Abstract

Logistic Regression is a well-known classification method that has been used widely in many applications of data mining, machine learning, computer vision, and bioinformatics. Sparse logistic regression embeds feature selection in the classification framework using the 1-norm regularization, and is attractive in many applications involving high-dimensional data. In this paper, we propose Lassplore for solving Large-scale sparse logistic regression. Specifically, we formulate the problem as the 1-ball constrained smooth convex optimization, and propose to solve the problem using the Nesterov's method, an optimal first-order black-box method for smooth convex optimization. One of the critical issues in the use of the Nesterov's method is the estimation of the step size at each of the optimization iterations. Previous approaches either applies the constant step size which assumes that the Lipschitz gradient is known in advance, or requires a sequence of decreasing step size which leads to slow convergence in practice. In this paper, we propose an adaptive line search scheme which allows to tune the step size adaptively and meanwhile guarantees the optimal convergence rate. Empirical comparisons with several state-of-theart algorithms demonstrate the efficiency of the proposed Lassplore algorithm for large-scale problems.

An efficient algorithm for a class of fused lasso problems

“…For the problem in (1), the associated proximal operator is the FLSA. The Nesterov's method has been applied to solve various sparse learning formulations [2,13,16,17,18,22,32].…”

Clinical risk prediction with multilinear sparse logistic regression

“…The resultant model is usually referred to as sparse logistic regression model [12]. Compared with the conventional logistic regression model obtained by minimizing Jorg, the w obtained by minimizing Jsp is sparse.…”

MultiScale Modeling of Islamic Organizations in UK

“…We formulate discriminative perspective mining problem in a general structured sparse learning framework [18]. The following steps describe our algorithm: 1) For each topic, calculate the frequency of the words occurring within a fixed sized window of the topic keyword.…”

“…The following steps describe our implementation: 1) For each topic relevant to a scale, calculate the frequency of the keywords co-occurring with the topic phrase in a document. 2) Use a sparse regression method with logistic loss discussed in previous section to learn the discriminant perspectives for each class using SLEP logistic sparse learning function [18]. 3) Use the identified perspectives to create a document x perspective matrix.…”

“…To compute the Newton direction is an 1 regularized least squares problem, which is also called Lasso [18]. It is time consuming for directly solving (5).…”