Communication‐Efficient Modeling with Penalized Quantile Regression for Distributed Data

Abstract

In order to deal with high-dimensional distributed data, this article develops a novel and communication-efficient approach for sparse and high-dimensional data with the penalized quantile regression. In each round, the proposed method only requires the master machine to deal with a sparse penalized quantile regression which could be realized fastly by proximal alternating direction method of multipliers (ADMM) algorithm and the other worker machines to compute the subgradient on local data. The advantage of the proximal ADMM algorithm is that it could make every parameter of iteration to have closed formula even in high-dimensional case, which greatly improves the speed of calculation. As for the communication efficiency, the proposed method does not sacrifice any statistical accuracy and provably improves the estimation error obtained by centralized method, provided the penalty levels are chosen properly. Moreover, the asymptotic properties of the proposed estimation and the convergence of the algorithm are convincible. Especially, it presents extensive experiments on both the numerical simulations and the HIV drug resistance data analysis, which all confirm the significant efficiency of our proposed method in quantile regression for distributed data by comparative and empirical analysis.

Distributed Sparse Feature Selection in Communication-Restricted Networks

“…We have also described it in details in Appendix A. We now describe our proposed method which is basically built upon the debiased LASSO estimator in (3). By (𝑋 𝑖 , 𝑌 𝑖 ), we represent the 𝑖th patch of data which is located in client 𝑖 ∈ [𝑁].…”

“…Consider the linear model in (1), and let θ𝑑 be as in (3). According to (4), we know that θ𝑑 |𝑋 ∼ N 𝜃 * + 𝑅, 𝜎 2 𝑛 𝑀 Σ𝑀 𝑇 with high probability, then we have:…”

“…This problem has been the center of attention in a variety of research areas [2], [3], [4], [5], [6], [7]. In this work, we set out to provide a solution for a particular case of interest: sparse linear feature selection in high dimensions when samples are distributed over a network of size 𝑁 ≥ 1 with at least one central node.…”