Collaborative Machine Learning at the Wireless Edge with Blind Transmitters

Amiri, Mohammad Mohammadi; Duman, Tolga M.; Gündüz, Deniz

doi:10.1109/globalsip45357.2019.8969185

Cited by 56 publications

(53 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case of fading channels [11], each node uses power control to cancel the channel effect at the receiver, where nodes that experience deep fading do not transmit. In [12], the authors extended the method for transmitting without knowing the channel state at the transmitter. The channel fading is mitigated at the receiver by using multiple antennas, where the fading diminishes as the number of antennas approaches infinity.…”

Section: B Distributed Learning Over Macmentioning

confidence: 99%

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

Sery

Cohen

2020

IEEE Trans. Signal Process.

149

114

View full text Add to dashboard Cite

We consider a distributed learning problem over multiple access channel (MAC) using a large wireless network. The computation is made by the network edge and is based on received data from a large number of distributed nodes which transmit over a noisy fading MAC. The objective function is a sum of the nodes' local loss functions. This problem has attracted a growing interest in distributed sensing systems, and more recently in federated learning. We develop a novel Gradient-Based Multiple Access (GBMA) algorithm to solve the distributed learning problem over MAC. Specifically, the nodes transmit an analog function of the local gradient using common shaping waveforms and the network edge receives a superposition of the analog transmitted signals used for updating the estimate. GBMA does not require power control or beamforming to cancel the fading effect as in other algorithms, and operates directly with noisy distorted gradients. We analyze the performance of GBMA theoretically, and prove that it can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks. Specifically, we establish a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient. Furthermore, we provide energy scaling laws for approaching the centralized convergence rate as the number of nodes increases. Finally, experimental results support the theoretical findings, and demonstrate strong performance of GBMA using synthetic and real data.Tomer Sery is with the

show abstract

Section: B Distributed Learning Over Macmentioning

confidence: 99%

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

Sery

Cohen

2020

IEEE Trans. Signal Process.

149

114

View full text Add to dashboard Cite

show abstract

“…In [11], [12], the bandwidth-efficient analog communication technique in [10] is combined with power control over a bandwidth-limited fading MAC, significantly reducing the communication load. Beamforming techniques at a multi-antenna PS for increasing the number of participating devices and overcoming the lack of CSI at the devices are introduced in [13] and [14], respectively. In [15], resource allocation across devices for FL over wireless channels is formulated as an optimization problem aiming to minimize the learning empirical loss function.…”

Section: Introductionmentioning

confidence: 99%

Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge

Amiri

Gündüz

Kulkarni

et al. 2021

IEEE Trans. Wireless Commun.

Self Cite

152

View full text Add to dashboard Cite

We study federated learning (FL) at the wireless edge, where power-limited devices with local datasets collaboratively train a joint model with the help of a remote parameter server (PS). We assume that the devices are connected to the PS through a bandwidth-limited shared wireless channel. At each iteration of FL, a subset of the devices are scheduled to transmit their local model updates to the PS over orthogonal channel resources, while each participating device must compress its model update to accommodate to its link capacity. We design novel scheduling and resource allocation policies that decide on the subset of the devices to transmit at each round, and how the resources should be allocated among the participating devices, not only based on their channel conditions, but also on the significance of their local model updates. We then establish convergence of a wireless FL algorithm with device scheduling, where devices have limited capacity to convey their messages. The results of numerical experiments show that the proposed scheduling policy, based on both the channel conditions and the significance of the local model updates, provides a better longterm performance than scheduling policies based only on either of the two metrics individually. Furthermore, we observe that when the data is independent and identically distributed (i.i.d.) across devices, selecting a single device at each round provides the best performance, while when the data distribution is noni.i.d., scheduling multiple devices at each round improves the performance. This observation is verified by the convergence result, which shows that the number of scheduled devices should increase for a less diverse and more biased data distribution.

show abstract

“…Accordingly, for fixed gradient values, each of the M interference terms in (45b) has zero mean and their variances scale with M−1 K . Thus, similar to the ideal case (where the receive chains are equipped with infinite resolution ADCs as considered in [12]), the interference term approaches zero as K → ∞. In other words, using a sufficiently large number of antennas at the PS eliminates the destructive effects of the interference on the learning process, and the estimate for the gradient vector is obtained as…”

Section: Dsgd With Low-resolution Adcs At the Psmentioning

confidence: 97%

“…where n ∈ [N +N cp ], L is the number of channel taps, τ mkl is the time delay and h t mkl ∈ C is the gain of the l-th channel tap from the m-th worker to the k-th antenna of the PS. Note that this is nothing but the machine learning over-the-air framework of [12]. We assume that h t mkl are zero-mean (circularly symmetric) complex Gaussian with…”

Section: System Modelmentioning

confidence: 99%

“…When the training data is randomly split among the workers, LFL with a small number of quantization levels performs as well as a system with unquantized parameters. In another line of research, [12] considers a federated learning system for which there is no CSI at the workers; hence the PS employs multiple antennas to align the received signals. In [13], this study is extended further, and a convergence analysis for the blind federated learning with both perfect and imperfect CSI is performed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Blind Federated Learning at the Wireless Edge With Low-Resolution ADC and DAC

Tegin

Duman

2021

IEEE Trans. Wireless Commun.

Self Cite

View full text Add to dashboard Cite

We study collaborative machine learning systems where a massive dataset is distributed across independent workers which compute their local gradient estimates based on their own datasets. Workers send their estimates through a multipath fading multiple access channel with orthogonal frequency division multiplexing to mitigate the frequency selectivity of the channel. We assume that there is no channel state information (CSI) at the workers, and the parameter server (PS) employs multiple antennas to align the received signals. To reduce the power consumption and the hardware costs, we employ complexvalued low-resolution digital-to-analog converters (DACs) and analog-to-digital converters (ADCs), at the transmitter and the receiver sides, respectively, and study the effects of practical low-cost DACs and ADCs on the learning performance. Our theoretical analysis shows that the impairments caused by lowresolution DACs and ADCs, including those of one-bit DACs and ADCs, do not prevent the convergence of the federated learning algorithms, and the multipath channel effects vanish when a sufficient number of antennas are used at the PS. We also validate our theoretical results via simulations, and demonstrate that using low-resolution, even one-bit, DACs and ADCs causes only a slight decrease in the learning accuracy.

show abstract

Collaborative Machine Learning at the Wireless Edge with Blind Transmitters

Cited by 56 publications

References 20 publications

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edge

Blind Federated Learning at the Wireless Edge With Low-Resolution ADC and DAC

Contact Info

Product

Resources

About