Distributed Ridge Regression with Feature Partitioning

Gratton, Cristiano; Venkategowda, Naveen K. D.; Arablouei, Reza; Werner, Stefan

doi:10.1109/acssc.2018.8645549

Cited by 15 publications

(9 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Learning over Distributed Features. Gratton et al (2018) applies ADMM to solve ridge regression. Ying et al (2018) proposes a stochastic learning method via variance reduction.…”

Section: Related Workmentioning

confidence: 99%

Learning Privately over Distributed Features: An ADMM Sharing Approach

Hu,

Liu,

Kong

et al. 2019

Preprint

View full text Add to dashboard Cite

Distributed machine learning has been widely studied in order to handle exploding amount of data. In this paper, we study an important yet less visited distributed learning problem where features are inherently distributed or vertically partitioned among multiple parties, and sharing of raw data or model parameters among parties is prohibited due to privacy concerns. We propose an ADMM sharing framework to approach risk minimization over distributed features, where each party only needs to share a single value for each sample in the training process, thus minimizing the data leakage risk. We establish convergence and iteration complexity results for the proposed parallel ADMM algorithm under non-convex loss. We further introduce a novel differentially private ADMM sharing algorithm and bound the privacy guarantee with carefully designed noise perturbation. The experiments based on a prototype system shows that the proposed ADMM algorithms converge efficiently in a robust fashion, demonstrating advantage over gradient based methods especially for data set with high dimensional feature spaces.

show abstract

“…Learning over Distributed Features. Gratton et al (2018) applies ADMM to solve ridge regression. Ying et al (2018) proposes a stochastic learning method via variance reduction.…”

Section: Related Workmentioning

confidence: 99%

Learning Privately over Distributed Features: An ADMM Sharing Approach

Hu,

Liu,

Kong

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…In this context, each agent in the network only possesses information of a local cost function and the agents aim to collaboratively minimize the sum of the local objective functions. Such optimization problems are relevant to several applications in statistics [3]- [5], signal processing [6]- [8] and control [1], [2].…”

Section: Introductionmentioning

confidence: 99%

“…There have been several works developing algorithms for solving distributed convex optimization problems over adhoc networks. However, many existing algorithms only offer solutions for problems with smooth objective functions, see, e.g., [5], [9], [10]. Distributed optimization problems with non-smooth objectives have been considered in [1], [2], [4], [11]- [16].…”

Section: Introductionmentioning

confidence: 99%

Distributed Learning with Non-Smooth Objective Functions

Gratton

Venkategowda

Arablouei

et al. 2021

2020 28th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

We develop a new distributed algorithm to solve a learning problem with non-smooth objective functions when data are distributed over a multi-agent network. We employ a zerothorder method to minimize the associated augmented Lagrangian in the primal domain using the alternating direction method of multipliers (ADMM) to develop the proposed algorithm, named distributed zeroth-order based ADMM (D-ZOA). Unlike most existing algorithms for non-smooth optimization, which rely on calculating subgradients or proximal operators, D-ZOA only requires function values to approximate gradients of the objective function. Convergence of D-ZOA to the centralized solution is confirmed via theoretical analysis and simulation results.

show abstract

“…Furthermore, collecting all the data in a fusion center creates a single point of failure. Therefore, it is imperative to develop algorithms that are capable of processing data spread across multiple agents [1][2][3][4][5][6][7].…”

Section: Introductionmentioning

confidence: 99%

“…Their relatively high computational complexity has partially motivated the works in [16][17][18][19][20]. While the approach of [16] is based on the average consensus strategy, the algorithms in [17][18][19][20][21] are based on diffusion strategies and, therefore, suffer from relatively slow convergence [6]. The convergence speed of the algorithm proposed in [13] greatly depends on the network topology and dimensionality of the data.…”

Section: Introductionmentioning

confidence: 99%

Consensus-based Distributed Total Least-squares Estimation Using Parametric Semidefinite Programming

Gratton

Venkategowda

Arablouei

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

We propose a new distributed algorithm to solve the total least-squares (TLS) problem when data are distributed over a multi-agent network. To develop the proposed algorithm, named distributed ADMM TLS (DA-TLS), we reformulate the TLS problem as a parametric semidefinite program and solve it using the alternating direction method of multipliers (ADMM). Unlike the existing consensus-based approaches to distributed TLS estimation, DA-TLS does not require careful tuning of any design parameter. Numerical experiments demonstrate that the DA-TLS converges to the centralized solution significantly faster than the existing consensus-based TLS algorithms.

show abstract

Distributed Ridge Regression with Feature Partitioning

Cited by 15 publications

References 29 publications

Learning Privately over Distributed Features: An ADMM Sharing Approach

Learning Privately over Distributed Features: An ADMM Sharing Approach

Distributed Learning with Non-Smooth Objective Functions

Consensus-based Distributed Total Least-squares Estimation Using Parametric Semidefinite Programming

Contact Info

Product

Resources

About