COTAF: Convergent Over-the-Air Federated Learning

Sery, Tomer; Shlezinger, Nir; Cohen, Kobi; Eldar, Yonina C.

doi:10.1109/globecom42002.2020.9322580

Cited by 23 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each coordinate is then transmitted over the Gaussian channel using one channel use per transmission. Similar scaling approaches are also presented in [38,39,[44][45][46]. On the other hand, digital schemes rely on gradient quantization and channel coding.…”

Section: Introductionmentioning

confidence: 88%

Fundamental limits of over-the-air optimization: Are analog schemes optimal?

Jha¹,

Mayekar²,

Tyagi³

2021

Preprint

View full text Add to dashboard Cite

We consider over-the-air convex optimization on a d−dimensional space where coded gradients are sent over an additive Gaussian noise channel with variance σ 2 . The codewords satisfy an average power constraint P , resulting in the signal-to-noise ratio (SNR) of P/σ 2 . We derive bounds for the convergence rates for over-the-air optimization. Our first result is a lower bound for the convergence rate showing that any code must slowdown the convergence rate by a factor of roughly d/ log(1 + SNR). Next, we consider a popular class of schemes called analog coding, where a linear function of the gradient is sent. We show that a simple scaled transmission analog coding scheme results in a slowdown in convergence rate by a factor of d(1 + 1/SNR). This matches the previous lower bound up to constant factors for low SNR, making the scaled transmission scheme optimal at low SNR. However, we show that this slowdown is necessary for any analog coding scheme. In particular, a slowdown in convergence by a factor of √ d for analog coding remains even when SNR tends to infinity. Remarkably, we present a simple quantize-and-modulate scheme that uses Amplitude Shift Keying and almost attains the optimal convergence rate at all SNRs.

show abstract

Section: Introductionmentioning

confidence: 88%

Fundamental limits of over-the-air optimization: Are analog schemes optimal?

Jha¹,

Mayekar²,

Tyagi³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In [12], the authors introduced update-importance-based client scheduling schemes to reduce the required number of model training rounds by selecting a subset of clients for local updates in each round of training. The authors of [13] proposed a convergent over-the-air FL scheme to reduce bandwidth and energy consumption by inducing precoding and scaling upon transmissions to gradually mitigate the effect of the noisy channel. In [14], the authors proposed a federated dropout scheme to enable FL on resource-constrained devices by tackling both the communication and computation resource bottlenecks.…”

Section: Related Workmentioning

confidence: 99%

Resource Consumption for Supporting Federated Learning in Wireless Networks

Liu

Qin

Sun

et al. 2022

IEEE Trans. Wireless Commun.

View full text Add to dashboard Cite

Federated learning (FL) has recently become one of the hottest focuses in wireless edge networks with the ever-increasing computing capability of user equipment (UE). In FL, UEs train local machine learning models and transmit them to an aggregator, where a global model is formed and then sent back to UEs. In wireless networks, local training and model transmission can be unsuccessful due to constrained computing resources, wireless channel impairments, bandwidth limitations, etc., which degrades FL performance in model accuracy and/or training time. Moreover, we need to quantify the benefits and cost of deploying edge intelligence, as model training and transmission consume certain amount of resources. Therefore, it is imperative to deeply understand the relationship between FL performance and multiple-dimensional resources. In this paper, we construct an analytical model to investigate the relationship between the FL model accuracy and consumed resources in FL empowered wireless edge networks. Based on the analytical model, we explicitly quantify the model accuracy, available computing resources and communication resources. Numerical results validate the effectiveness of our theoretical modeling and analysis, and demonstrate the trade-off between the communication and computing resources for achieving a certain model accuracy.

show abstract

“…Existing FL frameworks can be classified into synchronous FL and asynchronous FL according to model updating. Most of the studies on synchronous FL do not consider the issues of stragglers due to the device heterogeneity and the instability of network conditions [1] [15] [16]. To eliminate the straggler effect on the statistical heterogeneity, Li et al [17] propose a near term, experimentally prove this term can improve the stability of the framework and provide convergence guarantees in theory.…”

Section: B Related Workmentioning

confidence: 99%

CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework

Duan¹,

Liu²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is an emerging distributed machine learning paradigm that protects privacy and tackles the problem of isolated data islands. At present, there are two main communication strategies of FL: synchronous FL and asynchronous FL. The advantages of synchronous FL are that the model has high precision and fast convergence speed. However, this synchronous communication strategy has the risk that the central server waits too long for the devices, namely, the straggler effect which has a negative impact on some timecritical applications. Asynchronous FL has a natural advantage in mitigating the straggler effect, but there are threats of model quality degradation and server crash. Therefore, we combine the advantages of these two strategies to propose a clustered semi-asynchronous federated learning (CSAFL) framework. We evaluate CSAFL based on four imbalanced federated datasets in a non-IID setting and compare CSAFL to the baseline methods.The experimental results show that CSAFL significantly improves test accuracy by more than +5% on the four datasets compared to TA-FedAvg. In particular, CSAFL improves absolute test accuracy by +34.4% on non-IID FEMNIST compared to TA-FedAvg.

show abstract

COTAF: Convergent Over-the-Air Federated Learning

Cited by 23 publications

References 20 publications

Fundamental limits of over-the-air optimization: Are analog schemes optimal?

Fundamental limits of over-the-air optimization: Are analog schemes optimal?

Resource Consumption for Supporting Federated Learning in Wireless Networks

CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework

Contact Info

Product

Resources

About