Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT

Abstract

The rapidly expanding number of IoT devices is generating huge quantities of data, but public concern over data privacy means users are apprehensive to send data to a central server for Machine Learning (ML) purposes. The easilychanged behaviours of edge infrastructure that Software Defined Networking provides makes it possible to collate IoT data at edge servers and gateways, where Federated Learning (FL) can be performed: building a central model without uploading data to the server. FedAvg is a FL algorithm which has been the subject of much study, however it suffers from a large number of rounds to convergence with non-Independent, Identically Distributed (non-IID) client datasets and high communication costs per round. We propose adapting FedAvg to use a distributed form of Adam optimisation, greatly reducing the number of rounds to convergence, along with novel compression techniques, to produce Communication-Efficient FedAvg (CE-FedAvg). We perform extensive experiments with the MNIST/CIFAR-10 datasets, IID/non-IID client data, varying numbers of clients, client participation rates, and compression rates. These show CE-FedAvg can converge to a target accuracy in up to 6× less rounds than similarly compressed FedAvg, while uploading up to 3× less data, and is more robust to aggressive compression. Experiments on an edge-computing-like testbed using Raspberry Pi clients also show CE-FedAvg is able to reach a target accuracy in up to 1.7× less real time than FedAvg.

A Classification of the Enabling Techniques for Low Latency and Reliable Communications in 5G and Beyond: AI-Enabled Edge Caching

“…Also, the HierFAVG mechanism in [146] presented improved communication efficiency to outperform the FedAvg. Similarly, the CE-FedAvg mechanism in [147] presented improved communication efficiency to outperform the FedAvg. We present the details of the FedOpt, HierFAVG, and CE-FedAvg mechanisms in later paragraphs.…”

“…For instance, when dynamic and heterogeneous largescale IoT scenarios in [74], [85], [125], [142], [147] are considered, FL algorithms can be employed to enable the resource-constrained IoT devices to learn a shared learning model without centralizing the training data. However, the FL algorithms may achieve reduced accuracy compared to DRL algorithms, although the difference may be relatively negligible.…”

“…Therefore, [37], [90], [138], [145], [146]- [148] presented various mechanisms to address the challenge of massive communication overhead in FL. For instance, to address the challenges of the traditional FedAvg algorithm, novel FL frameworks were proposed in [138], [142], [146], [147]. The FedOpt framework in [138] was designed to address some challenges of the state-of-the-art FL frameworks.…”

“…To ensure improved performance, FedOpt designs a new data compression algorithm and then integrates an efficient encryption technique with differential privacy. Similarly, the HierFAVG framework in [146] and CE-FedAvg framework in [147] achieved improved communication efficiency. The main idea of the CE-FedAvg was to update the FedAvg algorithm by reducing both the number of rounds taken to converge to a given accuracy and the total data uploaded during training over FedAvg.…”

“…Moreover, more research is needed to improve the performance of FL frameworks as it was shown in [39], [43], [140] that although the frameworks are effective at reducing latency, they may be less effective at guaranteeing the required data privacy. Several issues regarding FL techniques were discussed in the recent work of [37], [39], [40], [90], [138], [142], [144], [146], [147], [152]- [154]. The work in [154] discussed the technical challenges of applying FL in vehicular IoT, and highlighted the necessary improvements that should be made for IoT technologies.…”