A Parallel Algorithm for Large-Scale Nonconvex Penalized Quantile Regression

Yu, Liangli; Lin, Nan; Wang, Lan

doi:10.1080/10618600.2017.1328366

Cited by 40 publications

(19 citation statements)

References 34 publications

(44 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, parallel algorithms have been applied to large scale penalized regresion, such as Liqun et al (2017) and Fan et al (2020), and achieved good performance in numerical experiments. Extending nested ADMM to parallel algorithms' framework is a potentially valuable work for big data.…”

Section: Discussionmentioning

confidence: 99%

Penalized and constrained LAD estimation in fixed and high dimension

Ran

Yang

2021

Stat Papers

View full text Add to dashboard Cite

Recently, many literatures have proved that prior information and structure in many application fields can be formulated as constraints on regression coefficients. Following these work, we propose a L 1 penalized LAD estimation with some linear constraints in this paper. Different from constrained lasso, our estimation performs well when heavy-tailed errors or outliers are found in the response. In theory, we show that the proposed estimation enjoys the Oracle property with adjusted normal variance when the dimension of the estimated coefficients p is fixed. And when p is much greater than the sample size n, the error bound of proposed estimation is sharper than k log( p)/n. It is worth noting the result is true for a wide range of noise distribution, even for the Cauchy distribution. In algorithm, we not only consider an typical linear programming to solve proposed estimation in fixed dimension , but also present an nested alternating direction method of multipliers (ADMM) in high dimension. Simulation and application to real data also confirm that proposed estimation is an effective alternative when constrained lasso is unreliable.

show abstract

Section: Discussionmentioning

confidence: 99%

Penalized and constrained LAD estimation in fixed and high dimension

Ran

Yang

2021

Stat Papers

View full text Add to dashboard Cite

show abstract

“…Comparison with ADMM. Recently, researchers have developed new optimization techniques based on alternating direction method of multiplier (ADMM) for solving QR problems (see, e.g., Yu, Lin and Wang (2017); Gu et al (2018)). We refer the readers to Boyd et al (2011) for more details on ADMM.…”

Section: In Particularmentioning

confidence: 99%

“…The primal update of β can be divided into two steps that fit a parallel computing scheme as suggested in Yu, Lin and Wang (2017). Following Yu, Lin and Wang (2017), in the g-th iteration, each of the primal β-step, r-step, and the dual step can all be updated separately for each batch of data by allowing the communication of the estimator of the (g − 1)-th iteration β (g−1) . For each iteration g and data batch k, we record the local computation time T (g) 1,k and aggregation time T (g) 2 .…”

Section: In Particularmentioning

confidence: 99%

Quantile regression under memory constraint

Chen¹,

Liu²,

Zhang³

2019

Ann. Statist.

123

View full text Add to dashboard Cite

This paper studies the inference problem in quantile regression (QR) for a large sample size n but under a limited memory constraint, where the memory can only store a small batch of data of size m. A natural method is the naïve divide-and-conquer approach, which splits data into batches of size m, computes the local QR estimator for each batch, and then aggregates the estimators via averaging. However, this method only works when n = o(m 2 ) and is computationally expensive. This paper proposes a computationally efficient method, which only requires an initial QR estimator on a small batch of data and then successively refines the estimator via multiple rounds of aggregations. Theoretically, as long as n grows polynomially in m, we establish the asymptotic normality for the obtained estimator and show that our estimator with only a few rounds of aggregations achieves the same efficiency as the QR estimator computed on all the data. Moreover, our result allows the case that the dimensionality p goes to infinity. The proposed method can also be applied to address the QR problem under distributed computing environment (e.g., in a large-scale sensor network) or for real-time streaming data.

show abstract

“…e quantile regression model selection procedure has been proved to have oracle property under some appropriate penalties [15,16,21]. Yu and Lin [22] and Yu et al [23] used the popular ADMM algorithm to solve the calculation problem of large-scale penalized quantile regression. Recently Chen et al [24] studied certain highdimensional distributed quantile regression.…”

Section: Introductionmentioning

confidence: 99%

Communication‐Efficient Modeling with Penalized Quantile Regression for Distributed Data

2021

Complexity

View full text Add to dashboard Cite

In order to deal with high-dimensional distributed data, this article develops a novel and communication-efficient approach for sparse and high-dimensional data with the penalized quantile regression. In each round, the proposed method only requires the master machine to deal with a sparse penalized quantile regression which could be realized fastly by proximal alternating direction method of multipliers (ADMM) algorithm and the other worker machines to compute the subgradient on local data. The advantage of the proximal ADMM algorithm is that it could make every parameter of iteration to have closed formula even in high-dimensional case, which greatly improves the speed of calculation. As for the communication efficiency, the proposed method does not sacrifice any statistical accuracy and provably improves the estimation error obtained by centralized method, provided the penalty levels are chosen properly. Moreover, the asymptotic properties of the proposed estimation and the convergence of the algorithm are convincible. Especially, it presents extensive experiments on both the numerical simulations and the HIV drug resistance data analysis, which all confirm the significant efficiency of our proposed method in quantile regression for distributed data by comparative and empirical analysis.

show abstract

A Parallel Algorithm for Large-Scale Nonconvex Penalized Quantile Regression

Cited by 40 publications

References 34 publications

Penalized and constrained LAD estimation in fixed and high dimension

Penalized and constrained LAD estimation in fixed and high dimension

Quantile regression under memory constraint

Communication‐Efficient Modeling with Penalized Quantile Regression for Distributed Data

Contact Info

Product

Resources

About