Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data

Ahn, Jinhyun; Simeone, Osvaldo; Joonhyuk, Kang

doi:10.48550/arxiv.1907.02745

Cited by 3 publications

(7 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where the first term in (12) is the minimum MSE achieved by a ∈ S i−1 from Lemma 1a, completing the proof.…”

Section: A Computation-optimal Policymentioning

confidence: 66%

“…where h k ∈ C is the channel coefficient between sensor k to the receiver and n is the additive white Gaussian noise (AWGN). It is assumed that the channel coefficients are known by both the sensors and the receiver, and the sensors' transmissions are well synchronized [10][11][12][13]15]. 4 We assume that the pre-processed signal…”

Section: Contributions and Paper Organizationmentioning

confidence: 99%

“…• Wireless distributed machine learning. In wireless machine learning applications that adopt distributed stochastic gradient descent algorithms for model training, each mobile user calculates the gradient of its local cost function of its own data set in terms of the model parameters and concurrently sends it to the parameter server (i.e., a base station), which then computes a weighted average of the gradients and broadcast it to the mobile users for further iteration until convergence [10][11][12].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Over-the-Air Computation Systems: Optimization, Analysis and Scaling Laws

Liu¹,

Zang²,

Li³

et al. 2019

Preprint

View full text Add to dashboard Cite

For future Internet of Things (IoT)-based Big Data applications (e.g., smart cities/transportation), wireless data collection from ubiquitous massive smart sensors with limited spectrum bandwidth is very challenging. On the other hand, to interpret the meaning behind the collected data, it is also challenging for edge fusion centers running computing tasks over large data sets with limited computation capacity. To tackle these challenges, by exploiting the superposition property of a multiple-access channel and the functional decomposition properties, the recently proposed technique, over-the-air computation (AirComp), enables an effective joint data collection and computation from concurrent sensor transmissions. In this paper, we focus on a single-antenna AirComp system consisting of K sensors and one receiver (i.e., the fusion center). We consider an optimization problem to minimize the computation meansquared error (MSE) of the K sensors' signals at the receiver by optimizing the transmitting-receiving (Tx-Rx) policy, under the peak power constraint of each sensor. Although the problem is not convex, we derive the computation-optimal policy in closed form. Also, we comprehensively investigate the ergodic performance of AirComp systems in terms of the average computation MSE and the average power consumption under Rayleigh fading channels with different Tx-Rx policies. For the computation-optimal policy, we prove that its average computation MSE has a decay rate of O(1/ √ K), and our numerical results illustrate that the policy also has a vanishing average power consumption with the increasing K, which jointly show the computation effectiveness and the energy efficiency of the policy with a large number of sensors.

show abstract

“…where the first term in (12) is the minimum MSE achieved by a ∈ S i−1 from Lemma 1a, completing the proof.…”

Section: A Computation-optimal Policymentioning

confidence: 66%

Section: Contributions and Paper Organizationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Over-the-Air Computation Systems: Optimization, Analysis and Scaling Laws

Liu¹,

Zang²,

Li³

et al. 2019

Preprint

View full text Add to dashboard Cite

show abstract

“…With the significant advance cloud computing, big data, and machine learning, intelligent transportation is a trend and requirement. As one of the most dominant distributed machine learning technologies, Federated Learning (FL) has been widely studied to deal with the data science recently [9]- [11]. The FL technology allows participant devices to collaboratively build a shard model while preserving privacy data locally [12].…”

Section: Introductionmentioning

confidence: 99%

Semi-asynchronous Hierarchical Federated Learning for Cooperative Intelligent Transportation Systems

Chen¹,

You²,

Jiang³

2021

Preprint

View full text Add to dashboard Cite

Cooperative Intelligent Transport System (C-ITS) is a promising network to provide safety, efficiency, sustainability, and comfortable services for automated vehicles and road infrastructures by taking advantages from participants. However, the components of C-ITS usually generate large amounts of data, which makes it difficult to explore data science. Currently, federated learning has been proposed as an appealing approach to allow users to cooperatively reap the benefits from trained participants. Therefore, in this paper, we propose a novel Semi-asynchronous Hierarchical Federated Learning (SHFL) framework for C-ITS that enables elastic edge to cloud model aggregation from data sensing. We further formulate a joint edge node association and resource allocation problem under the proposed SHFL framework to prevent personalities of heterogeneous road vehicles and achieve communication-efficiency. To deal with our proposed Mixed integer nonlinear programming (MINLP) problem, we introduce a distributed Alternating Direction Method of Multipliers (ADMM)-Block Coordinate Update (BCU) algorithm. With this algorithm, a tradeoff between training accuracy and transmission latency has been derived. Numerical results demonstrate the advantages of the proposed algorithm in terms of training overhead and model performance.

show abstract

“…Hence, it is necessary to optimize the delay for wireless FL implementation. Some of the challenges of FL over wireless networks have been studied in (Zhu et al, 2018b;Ahn et al, 2019;Yang et al, 2018;Zeng et al, 2019;Chen et al, 2019;Tran et al, 2019). To minimize latency, a broadband analog aggregation multi-access scheme for FL was designed in (Zhu et al, 2018b).…”

Section: Introductionmentioning

confidence: 99%

Delay Minimization for Federated Learning Over Wireless Communication Networks

Yang,

Chen,

Saad

et al. 2020

Preprint

View full text Add to dashboard Cite

In this paper, the problem of delay minimization for federated learning (FL) over wireless communication networks is investigated. In the considered model, each user exploits limited local computational resources to train a local FL model with its collected data and, then, sends the trained FL model parameters to a base station (BS) which aggregates the local FL models and broadcasts the aggregated FL model back to all the users. Since FL involves learning model exchanges between the users and the BS, both computation and communication latencies are determined by the required learning accuracy level, which affects the convergence rate of the FL algorithm. This joint learning and communication problem is formulated as a delay minimization problem, where it is proved that the objective function is a convex function of the learning accuracy. Then, a bisection search algorithm is proposed to obtain the optimal solution. Simulation results show that the proposed algorithm can reduce delay by up to 27.3% compared to conventional FL methods.

show abstract

Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data

Cited by 3 publications

References 0 publications

Over-the-Air Computation Systems: Optimization, Analysis and Scaling Laws

Over-the-Air Computation Systems: Optimization, Analysis and Scaling Laws

Semi-asynchronous Hierarchical Federated Learning for Cooperative Intelligent Transportation Systems

Delay Minimization for Federated Learning Over Wireless Communication Networks

Contact Info

Product

Resources

About