Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Abstract

The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment (e.g., dynamic channel and interference), limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources (e.g., computational power). This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.

Distributed Over-the-air Computing for Fast Distributed Optimization: Beamforming Design and Convergence Analysis

“…By substituting (11) in Proposition 1, the problem of alignment-error minimization over the alignment factor η is…”

“…Attempts to overcome the bottleneck have led to the proposal of different techniques including radio resource management [5], [6], model quantization [7], [8], and device scheduling [9], [10]. One particular class of techniques of our interest, known as over-the-air FEEL, features the application of AirComp to realize over-the-air model aggregation in FEEL [11]- [14]. The principle underpinning AirComp (as well as over-the-air FEEL) is to exploit the waveform superposition property such that the signal received at the server approximates a desired aggregation function of linear analog modulated data (e.g., local models/gradients) simultaneously transmitted by devices [12], [15].…”

Time-Correlated Sparsification for Efficient Over-the-Air Model Aggregation in Wireless Federated Learning

“…Federated edge learning (FEEL) refers to the implementation of federated learning algorithms [1], [2] in a wireless network, where edge devices train a shared machine learning (ML) model using their local datasets, and periodically communicate with a base station (BS) via wireless channels for global model aggregation. It is considered as a promising paradigm to facilitate edge intelligence, driving numerous applications such as Internet of things, augmented and virtual reality, self driving, and smart network management [3], [4].…”

Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying

“…In practical FL scenarios, some clients are stragglers and cannot send their updates regularly either because: (i) they cannot finish their computation within a prescribed deadline, or (ii) they cannot transmit their update to the PS successfully due to communication limitations [2]. Clients can suffer from intermittent connectivity to the PS, wherein their wireless communication channel is temporarily blocked [3][4][5][6][7][8][9].…”