A Joint Decentralized Federated Learning and Communications Framework for Industrial Networks

Savazzi, Stefano; Kianoush, Sanaz; Rampa, Vittorio; Bennis, Mehdi

doi:10.1109/camad50429.2020.9209305

Cited by 13 publications

(8 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Router can be a host or base-station. In mesh networks, further optimization via power control [18] may be also possible depending on the node deployment. Since devices do not need the router to relay information to the PS, which may be located in a different country, substantial energy savings are expected.…”

Section: Carbon Footprint Assessmentmentioning

confidence: 99%

“…The goal of the training task is to learn a ML model for the detection (classification) of the position of the human operators sharing the workspace, namely the human-robot distance d and the direction of arrival (DOA) θ. Further details about the the robotic manipulators, the industrial environment and the deployed sensors are given in [2], [18]. Input data x h , available online [24], are range-azimuth maps obtained from 3 time-division multiple-input-multiple output (TD-MIMO) frequency modulated continuous wave (FMCW) radars working in the 77 GHz band [22].…”

Section: A Case Study: Scenario-dependent Setupmentioning

confidence: 99%

See 1 more Smart Citation

A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning

Savazzi,

Kianoush,

Rampa

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Recent advances in distributed learning raise environmental concerns due to the large energy needed to train and move data to/from data centers. Novel paradigms, such as federated learning (FL), are suitable for decentralized model training across devices or silos that simultaneously act as both data producers and learners. Unlike centralized learning (CL) techniques, relying on big-data fusion and analytics located in energy hungry data centers, in FL scenarios devices collaboratively train their models without sharing their private data. This article breaks down and analyzes the main factors that influence the environmental footprint of FL policies compared with classical CL/Big-Data algorithms running in data centers. The proposed analytical framework takes into account both learning and communication energy costs, as well as the carbon equivalent emissions; in addition, it models both vanilla and decentralized FL policies driven by consensus. The framework is evaluated in an industrial setting assuming a real-world robotized workplace. Results show that FL allows remarkable end-to-end energy savings (30% ÷ 40%) for wireless systems characterized by low bit/Joule efficiency (50 kbit/Joule or lower). Consensusdriven FL does not require the parameter server and further reduces emissions in mesh networks (200 kbit/Joule). On the other hand, all FL policies are slower to converge when local data are unevenly distributed (often 2x slower than CL). Energy footprint and learning loss can be traded off to optimize efficiency.

show abstract

Section: Carbon Footprint Assessmentmentioning

confidence: 99%

Section: A Case Study: Scenario-dependent Setupmentioning

confidence: 99%

A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning

Savazzi,

Kianoush,

Rampa

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Most recently, researchers have studied distributed FEEL targeting a cluster of devices without coordination by a sever and connected by D2D links [16]- [18]. The original techniques for server-assisted FEEL can be adopted by arranging the devices to take turn or use orthogonal channels to play the role of edge server [17], [19]. As the resultant sequential aggregation is time consuming, attempts on realizing parallel operations have been made by selecting multiple weakly coupled clusters of devices to perform simultaneous intra-cluster aggregation [16], [18].…”

Section: Introductionmentioning

confidence: 99%

Distributed Over-the-air Computing for Fast Distributed Optimization: Beamforming Design and Convergence Analysis

Lin¹,

Gong²,

Huang³

2022

Preprint

View full text Add to dashboard Cite

Distributed optimization concerns the optimization of a common function in a distributed network, which finds a wide range of applications ranging from machine learning to vehicle platooning. Its key operation is to aggregate all local state information (LSI) at devices to update their states. The required extensive message exchange and many iterations cause a communication bottleneck when the LSI is high dimensional or at high mobility. To overcome the bottleneck, we propose in this work the framework of distributed over-the-air computing (AirComp) to realize a one-step aggregation for distributed optimization by exploiting simultaneous multicast beamforming of all devices and the property of analog waveform superposition of a multi-access channel. Equivalently, the technique superimposes multiple instances of conventional AirComp processes, giving rise to the challenge of jointly designing multicast beamforming at devices to rein in errors due to interference and channel distortion. We consider two design criteria. The first one is to minimize the sum AirComp error (i.e., sum mean-squared error (MSE)) with respect to the desired average-functional values. An efficient solution approach is proposed by transforming the non-convex beamforming problem into an equivalent concave-convex fractional program and solving it by nesting convex programming into a bisection search. The second criterion, called zero-forcing (ZF) multicast beamforming, is to force the received over-the-air aggregated signals at devices to be equal to the desired functional values. In this case, the optimal beamforming admits closed form. Both the MMSE and ZF beamforming exhibit a centroid structure resulting from averaging columns of conventional MMSE/ZF precoding. Last, the convergence of a classic distributed optimization algorithm is analyzed. The distributed AirComp is found to accelerate convergence by dramatically reducing communication latency. Another key finding is that the ZF beamforming outperforms the MMSE design as the latter is shown to cause bias in subgradient estimation.Z. Lin and K. Huang are with the Dept. of Electrical and Electronic Engr.

show abstract

“…The centralized FL also faces straggler's dilemma [21] due to the heterogeneity of edge devices, i.e., the FL training speed is limited by the devices with slowest computation and worst channel conditions. To overcome these challenges, device-to-device (D2D) communications based decentralized FL [22]- [24] was proposed, where each device only communicates with its neighbors. In particular, [24] considered digital transmission schemes in a joint learning and network simulation framework, to quantify the effects of model pruning, quantization and physical layer constraints for decentralized FL.…”

Section: Introductionmentioning

confidence: 99%

“…To overcome these challenges, device-to-device (D2D) communications based decentralized FL [22]- [24] was proposed, where each device only communicates with its neighbors. In particular, [24] considered digital transmission schemes in a joint learning and network simulation framework, to quantify the effects of model pruning, quantization and physical layer constraints for decentralized FL. Due to limited wireless bandwidth resources, the authors in [22], [23] proposed a decentralized stochastic gradient descent (DSGD) algorithm to improve the convergence performance in decentralized FL, where AirComp based D2D communication was developed to facilitate the consensus phase.…”

Section: Introductionmentioning

confidence: 99%

Over-the-Air Decentralized Federated Learning

Shi¹,

Zhou²,

Shi³

2021

Preprint

View full text Add to dashboard Cite

In this paper, we consider decentralized federated learning (FL) over wireless networks, where over-the-air computation (AirComp) is adopted to facilitate the local model consensus in a device-to-device (D2D) communication manner. However, the AirComp-based consensus phase brings the additive noise in each algorithm iterate and the consensus needs to be robust to wireless network topology changes, which introduce a coupled and novel challenge of establishing the convergence for wireless decentralized FL algorithm. To facilitate consensus phase, we propose an AirComp-based DSGD with gradient tracking and variance reduction (DSGT-VR) algorithm, where both precoding and decoding strategies are developed for D2D communication. Furthermore, we prove that the proposed algorithm converges linearly and establish the optimality gap for strongly convex and smooth loss functions, taking into account the channel fading and noise. The theoretical result shows that the additional error bound in the optimality gap depends on the number of devices. Extensive simulations verify the theoretical results and show that the proposed algorithm outperforms other benchmark decentralized FL algorithms over wireless networks.

show abstract

A Joint Decentralized Federated Learning and Communications Framework for Industrial Networks

Cited by 13 publications

References 14 publications

A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning

A Framework for Energy and Carbon Footprint Analysis of Distributed and Federated Edge Learning

Distributed Over-the-air Computing for Fast Distributed Optimization: Beamforming Design and Convergence Analysis

Over-the-Air Decentralized Federated Learning

Contact Info

Product

Resources

About