On the Tradeoff between Energy, Precision, and Accuracy in Federated Quantized Neural Networks

Abstract

Deploying federated learning (FL) over wireless networks with resource-constrained devices requires balancing between accuracy, energy efficiency, and precision. Prior art on FL often requires devices to train deep neural networks (DNNs) using a 32-bit precision level for data representation to improve accuracy. However, such algorithms are impractical for resourceconstrained devices since DNNs could require execution of millions of operations. Thus, training DNNs with a high precision level incurs a high energy cost for FL. In this paper, a quantized FL framework, that represents data with a finite level of precision in both local training and uplink transmission, is proposed.Here, the finite level of precision is captured through the use of quantized neural networks (QNNs) that quantize weights and activations in fixed-precision format. In the considered FL model, each device trains its QNN and transmits a quantized training result to the base station. Energy models for the local training and the transmission with the quantization are rigorously derived. An energy minimization problem is formulated with respect to the level of precision while ensuring convergence. To solve the problem, we first analytically derive the FL convergence rate and use a line search method. Simulation results show that our FL framework can reduce energy consumption by up to 53% compared to a standard FL model. The results also shed light on the tradeoff between precision, energy, and accuracy in FL over wireless networks.

Provision for Energy: A Resource Allocation Problem in Federated Learning for Edge Systems

“…FL avoids the need for data uploads and enables rapid access to real-time data, thus reducing pressure on communication resources and lowering service latency. It is a promising distributed learning algorithm that is likely to be applied in future internet of things systems [10][11][12][13][14][15].…”

Provision for Energy: Federated Learning in Edge Systems

“…The spectrum resource are even scarce and only a few edge devices are permitted to upload the trained local models in each round. Moreover, the limited battery lives of the edge devices increasingly attract the attention and the en-ergy consumption caused by communication is Therefore, the FL framework is one of the most promising distributed learning algorithms that will be applied to future Internet of Things (IoT) systems [12,13,14,15,16,17].…”