Federated Learning with Proximal Stochastic Variance Reduced Gradient Algorithms

Dinh, Canh T.; Tran, Nguyen H.; Nguyen, Tuan Dung; Bao, Wei; Zomaya, Albert Y.; Zhou, Bing Bing

doi:10.1145/3404397.3404457

Cited by 34 publications

(63 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Through the collaboration between model parameters in V , penalty terms P(V ) are computed, improving the performance of the c n under heterogeneous dataset D n . Notice that the core of PFL is the collaborative strategy among the clients, while the penalty term aims at deep optimization of ν n in [14], [16], [20], [22], which implies us to design fusion rule with considering data heterogeneity and model overfitting in this paper for each client. Moreover, when designing the fusion rule in this paper, each client's layer parameter is proposed to be viewed as the basic unit.…”

Section: Problem Formulationmentioning

confidence: 99%

“…Where the local step l λ = 1 and the learning rate η = 0.005. 4) pFedMe introduces the idea of personalization [20], which transforms the optimization problem into a bi-level decoupling problem from client personalization loss to global loss. Where the penalty coefficient λ and the number of personalized training steps K are set to 15 and 5, respectively.…”

Section: A Experimental Setupmentioning

confidence: 99%

“…In this case, some personalized W. Z. Yang, B. Chen, Y. J. Shen, J. Liu and L. Yu are with the Department of Automation, Zhejiang University of Technology, Hangzhou 310023, China, and also with the Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023, China (Correspondence email:bchen@aliyun.com). federated learning (PFL) methods have been proposed by [15]- [20], which are dedicated to optimising the collaboration strategy between client models. Specifically, a similar strategy to [14] is used in [19], with the improved aspect of changing the global model as the penalty target to a private model for each client.…”

Section: Introductionmentioning

confidence: 99%

“…Specifically, a similar strategy to [14] is used in [19], with the improved aspect of changing the global model as the penalty target to a private model for each client. Further, the pFedMe proposed in [20] uses the Moreau envelopes as a constraint term to enable personalized deployment of the client model, where a bi-level strategy was adopted to calculate the server model of conditional convergence. Based on the strategy of adding additional terms, the improved PFL method compensates for the weakness of personalized data processing in [8], which was caused by pursuing the global optimum through the weighted average client model.…”

Section: Introductionmentioning

confidence: 99%

“…3) The layer function-based fusion threshold rule is applied to the multi-layer multi-fusion strategy framework to improve personalized federated learning performance. Then, the extensive experiments on three benchmark datasets show that the proposed personalized federated learning based on the cross-fusion rule (pFedCFR) outperforms state-of-the-art (SOTA) PFL methods [19], [20], [22] and generic federated learning strategy [8], [14].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Cross-Fusion Rule for Personalized Federated Learning

Yang

Chen

Shen

et al. 2023

Preprint

View full text Add to dashboard Cite

Section: Problem Formulationmentioning

confidence: 99%

Section: A Experimental Setupmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Cross-Fusion Rule for Personalized Federated Learning

Yang

Chen

Shen

et al. 2023

Preprint

View full text Add to dashboard Cite

pvFed: Personalized Vertical Federated learning for Client‐Specific Tasks

Nishikawa,

Yanabe,

Sakuma

et al. 2024

IEEJ Transactions Elec Engng

View full text Add to dashboard Cite

Federated Learning (FL) is a distributed machine learning paradigm that enables multiple data holders to collaborate on building machine learning models while preserving the privacy of their data. FL can be categorized as horizontal or vertical, depending on the distribution characteristics of the data. Specifically, horizontal FL uses data partitioned in the sample space, whereas vertical FL uses data partitioned in the feature space. Traditional vertical FL methods aim to facilitate collaboration among clients to infer a single global target. However, these methods may be impractical because each client often has a unique target to be inferred. In this paper, we propose a novel vertical FL method, called personalized vertical federated learning (pvFed), which addresses this limitation by allowing each client to perform inferences specific to their individual task. To the best of our knowledge, no existing method currently addresses this limitation. The objective of pvFed is to construct a global model that generates a representation vector to support client inference. The global model, constructed using distillation and dimensionality reduction, takes a sample ID common to all clients as input and outputs a sample‐specific representation vector. Clients utilize the intermediate representation of their own model and the representation vectors output by the global model for inference. Because these vectors are not dependent on client‐specific tasks, clients can repurpose them for any additional tasks. Our experiments, conducted on two distinct data types—image and tabular data sets, under a vertical partitioning where each client had its own specific task, demonstrated the efficacy of vectors generated by the global model in pvFed for client inference. © 2024 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

show abstract

Federated learning with superquantile aggregation for heterogeneous data

et al. 2023

View full text Add to dashboard Cite

We present a federated learning framework that is designed to robustly deliver good predictive performance across individual clients with heterogeneous data. The proposed approach hinges upon a superquantile-based learning objective that captures the tail statistics of the error distribution over heterogeneous clients. We present a stochastic training algorithm that interleaves differentially private client filtering with federated averaging steps. We prove finite time convergence guarantees for the algorithm: O(1∕ √ T) in the nonconvex case in T communication rounds and O(exp(−T∕ 3∕2 ) + ∕T) in the strongly convex case with local condition number . Experimental results on benchmark datasets for federated learning demonstrate that our approach is competitive with classical ones in terms of average error and outperforms them in terms of tail statistics of the error.

show abstract

Federated Learning with Proximal Stochastic Variance Reduced Gradient Algorithms

Cited by 34 publications

References 10 publications

Cross-Fusion Rule for Personalized Federated Learning

Cross-Fusion Rule for Personalized Federated Learning

pvFed: Personalized Vertical Federated learning for Client‐Specific Tasks

Federated learning with superquantile aggregation for heterogeneous data

Contact Info

Product

Resources

About