Over-the-Air Federated Learning From Heterogeneous Data

Sery, Tomer; Shlezinger, Nir; Cohen, Kobi; Eldar, Yonina C.

doi:10.1109/tsp.2021.3090323

Cited by 157 publications

(69 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, Assumption 3 on the bounded variance is standard in the stochastic optimization literature [52]. As an aside, these assumptions are made for analytical tractability [48]. In fact, many more complicated learning models that do not satisfy these assumptions can also perform well in practice, which is validated by the experiments in Section V.…”

Section: Convergence Analysis and Problem Formulation A Basic Assumptionmentioning

confidence: 88%

“…By taking advantage of the SIC technique at the BS side, the signals from strong users can be decoded one by one to remove the co-channel interference for weak users [5], [23]. In the sequel, the residual interference composed of signals from weak users can be harnessed for function computation [46]- [48]. To actualize this design, we propose a successive signal processing (SSP) scheme taking full advantage of the superposition property of multiple access channels.…”

Section: Successive Signal Processingmentioning

confidence: 99%

“…Note that the above assumptions make the convergence analysis of AirFL tractable without loss of generality. Assumption 1 on the Lipschitz smoothness holds for a broad range of typical loss functions, such as the linear regression, cross entropy, logistic regression, and softmax classifier [20], [48]- [51]. Assumption 2 on the strong convexity can be easily extended to the Polyak-Lojasiewicz (PL) condition given in Appendix B.…”

Section: Convergence Analysis and Problem Formulation A Basic Assumptionmentioning

confidence: 99%

See 2 more Smart Citations

STAR-RIS Enabled Heterogeneous Networks: Ubiquitous NOMA Communication and Pervasive Federated Learning

Ni,

Liu,

Eldar

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper integrates non-orthogonal multiple access (NOMA) and over-the-air federated learning (AirFL) into a unified framework using one simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS). The STAR-RIS plays an important role in adjusting the decoding order of hybrid users for efficient interference mitigation and omni-directional coverage extension. To capture the impact of non-ideal wireless channels on AirFL, a closed-form expression for the optimality gap (a.k.a. convergence upper bound) between the actual loss and the optimal loss is derived. This analysis reveals that the learning performance is significantly affected by active and passive beamforming schemes as well as wireless noise. Furthermore, when the learning rate diminishes as the training proceeds, the optimality gap is explicitly characterized to converge with a linear rate. To accelerate convergence while satisfying QoS requirements, a mixed-integer non-linear programming (MINLP) problem is formulated by jointly designing the transmit power at users and the configuration mode of STAR-RIS. Next, a trust region-based successive convex approximation method and a penalty-based semidefinite relaxation approach are proposed to handle the decoupled non-convex subproblems iteratively. An alternating optimization algorithm is then developed to find a suboptimal solution for the original MINLP problem. Extensive simulation results show that i) the proposed framework can efficiently support NOMA and AirFL users via concurrent uplink communications, ii) our algorithms can achieve a faster convergence rate on IID and non-IID settings as compared to existing baselines, and iii) both the spectrum efficiency and learning performance can be significantly improved with the aid of the well-tuned STAR-RIS.

show abstract

Section: Convergence Analysis and Problem Formulation A Basic Assumptionmentioning

confidence: 88%

Section: Successive Signal Processingmentioning

confidence: 99%

Section: Convergence Analysis and Problem Formulation A Basic Assumptionmentioning

confidence: 99%

See 1 more Smart Citation

STAR-RIS Enabled Heterogeneous Networks: Ubiquitous NOMA Communication and Pervasive Federated Learning

Ni,

Liu,

Eldar

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We adopt in this paper the precoding scheme proposed in [23], where every client n precodes its model update m n t as…”

Section: A Analog Over-the-air Flmentioning

confidence: 99%

Robust Federated Learning via Over-The-Air Computation

Sifaou¹,

Li²

2021

Preprint

View full text Add to dashboard Cite

This paper investigates the robustness of over-theair federated learning to Byzantine attacks. The simple averaging of the model updates via over-the-air computation makes the learning task vulnerable to random or intended modifications of the local model updates of some malicious clients. We propose a robust transmission and aggregation framework to such attacks while preserving the benefits of over-the-air computation for federated learning. For the proposed robust federated learning, the participating clients are randomly divided into groups and a transmission time slot is allocated to each group. The parameter server aggregates the results of the different groups using a robust aggregation technique and conveys the result to the clients for another training round. We also analyze the convergence of the proposed algorithm. Numerical simulations confirm the robustness of the proposed approach to Byzantine attacks.

show abstract

“…Federated learning was introduced by [54] to handle distributed on-client learning [37,49,28]. A plethora of recent extensions have also been proposed [84,72,55,81,56,74,35,73,16]. Our approach of addressing the statistical heterogeneity by proposing a new objective, which is broadly applicable in these settings.…”

Section: Related Workmentioning

confidence: 99%

Federated Learning with Superquantile Aggregation for Heterogeneous Data

Pillutla¹,

Laguel²,

Malick³

et al. 2021

Preprint

View full text Add to dashboard Cite

We present a federated learning framework that is designed to robustly deliver good predictive performance across individual clients with heterogeneous data. The proposed approach hinges upon a superquantile-based learning objective that captures the tail statistics of the error distribution over heterogeneous clients. We present a stochastic training algorithm which interleaves differentially private client reweighting steps with federated averaging steps. The proposed algorithm is supported with finite time convergence guarantees that cover both convex and non-convex settings. Experimental results on benchmark datasets for federated learning demonstrate that our approach is competitive with classical ones in terms of average error and outperforms them in terms of tail statistics of the error.

show abstract

Over-the-Air Federated Learning From Heterogeneous Data

Cited by 157 publications

References 21 publications

STAR-RIS Enabled Heterogeneous Networks: Ubiquitous NOMA Communication and Pervasive Federated Learning

STAR-RIS Enabled Heterogeneous Networks: Ubiquitous NOMA Communication and Pervasive Federated Learning

Robust Federated Learning via Over-The-Air Computation

Federated Learning with Superquantile Aggregation for Heterogeneous Data

Contact Info

Product

Resources

About