Blind Federated Edge Learning

Abstract

We study federated edge learning (FEEL), where wireless edge devices, each with its own dataset, learn a global model collaboratively with the help of a wireless access point acting as the parameter server (PS). At each iteration, wireless devices perform local updates using their local data and the most recent global model received from the PS, and send their local updates to the PS over a wireless fading multiple access channel (MAC). The PS then updates the global model according to the signal received over the wireless MAC, and shares it with the devices. Motivated by the additive nature of the wireless MAC, we propose an analog 'over-the-air' aggregation scheme, in which the devices transmit their local updates in an uncoded fashion.However, unlike recent literature on over-the-air FEEL, here we assume that the devices do not have channel state information (CSI), while the PS has imperfect CSI. On the other hand, the PS is equipped with multiple antennas to alleviate the destructive effect of the channel, exacerbated due to the lack of perfect CSI. We design a receive beamforming scheme at the PS, and show that it can compensate for the lack of perfect CSI when the PS has a sufficient number of antennas. We also derive the convergence rate of the proposed algorithm highlighting the impact of the lack of perfect CSI, as well as the number of PS antennas. Both the experimental results and the convergence analysis illustrate the performance improvement of the proposed algorithm with the number of PS antennas, where the wireless fading MAC becomes deterministic despite the lack of perfect CSI when the PS has a sufficiently large number of antennas.

Blind Federated Learning at the Wireless Edge With Low-Resolution ADC and DAC

“…To perform the demodulation, we take the DFT of ( 10) which gives (13) where Q m [i] is the DFT of the quantization distortion noise and H mk [i]'s are the channel gains from the m-th worker to the k-th receive chain for the i-th subcarrier. H mk [i]'s are given by…”

“…In another line of research, [12] considers a federated learning system for which there is no CSI at the workers; hence the PS employs multiple antennas to align the received signals. In [13], this study is extended further, and a convergence analysis for the blind federated learning with both perfect and imperfect CSI is performed.…”

“…• Different from previous works regarding federated learning reviewed above ( [3]- [5], [8]- [13]), we consider a realistic wireless channel model where the channel between the workers and PS is modeled as a multipath fading MAC. • To cope with the realistic channel impairments, we transmit the local gradients using OFDM with a cyclic prefix (CP) to mitigate the ISI caused by the multipath.…”

Hierarchical Over-the-Air Federated Edge Learning

“…To address the communication cost concerns, over-the-air (OTA) aggregation [4] has become a popular method in wireless schemes thanks to its efficient strategy that allocates all the users to the same bandwidth, thereby handling the transmission and aggregation of the gradient updates simultaneously (over the air). For this framework, one approach to deal with the channel effects (particularly when there is no transmit side channel state information) is to increase the number of receive antennas at the PS [5]. Nevertheless, the disparity among the channel gains is still a critical factor when some MUs are far away from the PS.…”

“…Recent developments on FL include device selection algorithms [6], efficient communication schemes [4], [7]- [11], heterogeneity of data [12], and power and latency analysis [13], [14]. Although Federated Averaging [2] is the most common way to perform global aggregation in error-free setups, OTA communication has been preferred for wireless FL [5], [12], [15]. Furthermore, hierarchical federated learning (HFL) has been gaining increasing attention, where the objective is to utilize intermediate servers (IS) to form clusters to reduce communication costs.…”