Toward Energy-Efficient Federated Learning Over 5G+ Mobile Devices

Shi, Dian; Li, Liang; Chen, Rui; Prakash, Pavana; Pan, Miao; Fang, Yuguang

doi:10.1109/mwc.003.2100028

Cited by 33 publications

(10 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FL has become the most prevalent distributed learning framework due to its advantages in data privacy and parallel model training. However, since FL is resource-hungry, the limited communication and computing capabilities at client devices are the bottlenecks [8]. To mitigate this issue, there are two research directions to be explored: One is to design judicious resource scheduling and transmission schemes, including client selection [20,21], resource allocation [22][23][24][25], and hierarchical model aggregation [26,27].…”

Section: Related Workmentioning

confidence: 99%

“…To mitigate this issue, there are two research directions to be explored: One is to design judicious resource scheduling and transmission schemes, including client selection [20,21], resource allocation [22][23][24][25], and hierarchical model aggregation [26,27]. The other is to integrate model compression frameworks [8,9]. However, the latter introduces inevitable approximation errors.…”

Section: Related Workmentioning

confidence: 99%

“…For instance, the popular image recognition model VGG16 [7] comprises 138 million parameters (528MB in 32bit float) and requires 15.5G MACs (multiply-add computation) for forward propagation (FP) process, essentially making exclusive on-device training impractical for resourceconstrained mobile or IoT devices. One promising solution is to compress the ML model to lower both communication and computing workload [8][9][10]. However, model compression/quantization inevitably induces more inference/learning errors due to the low-precision calculations.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks

Zheng¹,

Zhu²,

Deng³

et al. 2023

Preprint

View full text Add to dashboard Cite

The increasingly deeper neural networks hinder the democratization of privacy-enhancing distributed learning, such as federated learning (FL), to resource-constrained devices. To overcome this challenge, in this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL), allowing multiple client devices to offload substantial training workloads to an edge server via layer-wise model split. By observing that existing PSL schemes incur excessive training latency and large volume of data transmissions, we propose an innovative PSL framework, namely, efficient parallel split learning (EPSL), to accelerate model training. To be specific, EPSL parallelizes client-side model training and reduces the dimension of activations' gradients for back propagation (BP) via last-layer gradient aggregation, leading to a significant reduction in server-side training and communication latency. Moreover, by considering the heterogeneous channel conditions and computing capabilities at client devices, we jointly optimize subchannel allocation, power control, and cut layer selection to minimize the per-round latency. Simulation results show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy compared with the stateof-the-art benchmarks, and the tailored resource management and layer split strategy can considerably reduce latency than the counterpart without optimization.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks

Zheng¹,

Zhu²,

Deng³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, much attention has been paid to the energy-efficient FL over mobile devices, where several advanced techniques are utilized to save energy during the FL training [20]. On the one hand, gradient sparsification [21], [22] and gradient quantization [23], [24] techniques can compress model updates in the transmission process, significantly reducing the communication burdens [25].…”

Section: Related Workmentioning

confidence: 99%

Energy and Spectrum Efficient Federated Learning via High-Precision Over-the-Air Computation

Li¹,

Huang²,

Shi³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Federated learning (FL) enables mobile devices to collaboratively learn a shared prediction model while keeping data locally. However, there are two major research challenges to practically deploy FL over mobile devices: (i) frequent wireless updates of huge size gradients v.s. limited spectrum resources, and (ii) energy-hungry FL communication and local computing during training v.s. batteryconstrained mobile devices. To address those challenges, in this paper, we propose a novel multibit over-the-air computation (M-AirComp) approach for spectrum-efficient aggregation of local model updates in FL and further present an energy-efficient FL design for mobile devices. Specifically, a high-precision digital modulation scheme is designed and incorporated in the M-AirComp, allowing mobile devices to upload model updates at the selected positions simultaneously in the multi-access channel. Moreover, we theoretically analyze the convergence property of our FL algorithm. Guided by FL convergence analysis, we formulate a joint transmission probability and local computing control

show abstract

“…This process is repeated for multiple communication rounds until the global model converges with a satisfactory accuracy. Although FL only requires the transmission of model updates between edge devices and the server instead of raw data, such model transfer can become a communication bottleneck, especially when dealing with modern deep neural networks (DNNs) that has a huge number of parameters (e.g., on the order of hundreds MB, or even GB) [2]. Additionally, the transmit power of edge devices is often limited in FL.…”

Section: Introductionmentioning

confidence: 99%

Energy-efficient Federated Edge Learning for Internet of Vehicles via Rate-Splitting Multiple Access

Zhang

Yeung

2022

2022 International Symposium on Wireless Communication Systems (ISWCS)

View full text Add to dashboard Cite

Federated Learning (FL) is a collaborative learning framework that enables edge devices to collaboratively learn a global model while keeping raw data locally. Although FL avoids leaking direct information from local datasets, sensitive information can still be inferred from the shared models. To address the privacy issue in FL, differential privacy (DP) mechanisms are leveraged to provide formal privacy guarantee. However, when deploying FL at the wireless edge with over-the-air computation, ensuring client-level DP faces significant challenges. In this paper, we propose a novel wireless FL scheme called private federated edge learning with sparsification (PFELS) to provide client-level DP guarantee with intrinsic channel noise while reducing communication and energy overhead and improving model accuracy. The key idea of PFELS is for each device to first compress its model update and then adaptively design the transmit power of the compressed model update according to the wireless channel status without any artificial noise addition. We provide a privacy analysis for PFELS and prove the convergence of PFELS under general non-convex and non-IID settings. Experimental results show that compared with prior work, PFELS can improve the accuracy with the same DP guarantee and save communication and energy costs simultaneously.

show abstract

Toward Energy-Efficient Federated Learning Over 5G+ Mobile Devices

Cited by 33 publications

References 8 publications

Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks

Efficient Parallel Split Learning over Resource-constrained Wireless Edge Networks

Energy and Spectrum Efficient Federated Learning via High-Precision Over-the-Air Computation

Energy-efficient Federated Edge Learning for Internet of Vehicles via Rate-Splitting Multiple Access

Contact Info

Product

Resources

About