Partial Synchronization to Accelerate Federated Learning Over Relay-Assisted Edge Networks

Qu, Zhihao; Guo, Song; Wang, Haozhao; Ye, Baoliu; Wang, Yi; Zomaya, Albert Y.; Tang, Bin

doi:10.1109/tmc.2021.3083154

Cited by 30 publications

(17 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This process is repeated until no neighbors are available in the designed clustering, and this is summarized in lines 21-25 of Algorithm 1. Meanwhile, the BS calculates the FL time of the scheduled devices according to (17) to check the feasibility of the scheduled local devices. Next, for each scheduled device, the BS updates the variable f n .…”

Section: Development and Properties Of The Proposed Fl-eocd Algorithm...mentioning

confidence: 99%

Decentralized Aggregation for Energy-Efficient Federated Learning via Overlapped Clustering and D2D Communications

Al-Abiad¹,

Obeed²,

Hossain³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has emerged as a distributed machine learning (ML) technique to train models without sharing users' private data. In this paper, we propose a decentralized FL scheme that is called federated learning empowered overlapped clustering for decentralized aggregation (FL-EOCD). The introduced FL-EOCD leverages deviceto-device (D2D) communications and overlapped clustering to enable decentralized aggregation, where a cluster is defined as a coverage zone of a typical device. The devices located on the overlapped clusters are called bridge devices (BDs). In the proposed FL-EOCD scheme, a clustering topology is envisioned where clusters are connected through BDs, so as the aggregated models of each cluster is disseminated to the other clusters in a decentralized manner without the need for a global aggregator or an additional hop of transmission. Unlike the star-based FL, the proposed FL-EOCD scheme involves a large number of local devices by reusing the RRBs in different non-adjacent clusters. To evaluate our proposed FL-EOCD scheme as opposed to baseline FL schemes, we consider minimizing the overall energyconsumption of devices while maintaining the convergence rate of FL subject to its time constraint. To this end, a joint optimization problem, considering scheduling the local devices/BDs to the CHs and computation frequency allocation, is formulated, where an iterative solution to this joint problem is devised. Extensive simulations are conducted to verify the effectiveness of the proposed FL-EOCD algorithm over FL conventional schemes in terms of energy consumption, latency, and convergence rate.

show abstract

Section: Development and Properties Of The Proposed Fl-eocd Algorithm...mentioning

confidence: 99%

Decentralized Aggregation for Energy-Efficient Federated Learning via Overlapped Clustering and D2D Communications

Al-Abiad¹,

Obeed²,

Hossain³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The popular FL conventional architectures are: (i) starbased FL architecture [8]- [13] and (ii) relay-assisted/dual-hop FL [14]- [17], also called hierarchical FL. In each iteration of the star-based FL, each device trains a local model based on its own dataset and then transmits its local trained model to the BS.…”

Section: A Federated Learning At the Network Edgementioning

confidence: 99%

Decentralized Aggregation for Energy-Efficient Federated Learning via Overlapped Clustering and D2D Communications

Al-Abiad¹,

Obeed²,

Hossain³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>Federated learning (FL) has emerged as a distributed machine learning (ML) technique to train models without sharing users' private data. In this paper, we propose a decentralized FL scheme that is called \underline{f}ederated \underline{l}earning \underline{e}mpowered \underline{o}verlapped \underline{c}lustering for \underline{d}ecentralized aggregation (FL-EOCD). The introduced FL-EOCD leverages device-to-device (D2D) communications and overlapped clustering to enable decentralized aggregation, where a cluster is defined as a coverage zone of a typical device. The devices located on the overlapped clusters are called bridge devices (BDs). In the proposed FL-EOCD scheme, a clustering topology is envisioned where clusters are connected through BDs, so as the aggregated models of each cluster is disseminated to the other clusters in a decentralized manner without the need for a global aggregator or an additional hop of transmission. Unlike the star-based FL, the proposed FL-EOCD scheme involves a large number of local devices by reusing the RRBs in different non-adjacent clusters. To evaluate our proposed FL-EOCD scheme as opposed to baseline FL schemes, we consider minimizing the overall energy-consumption of devices while maintaining the convergence rate of FL subject to its time constraint. To this end, a joint optimization problem, considering scheduling the local devices/BDs to the CHs and computation frequency allocation, is formulated, where an iterative solution to this joint problem is devised. Extensive simulations are conducted to verify the effectiveness of the proposed FL-EOCD algorithm over FL conventional schemes in terms of energy consumption, latency, and convergence rate.</p>

show abstract

“…However, discarding certain amount of devices during the training process may degrade the performance of the wireless FL systems due to the insufficient training data exploitation [24]. Therefore, to fully exploit all the available data, one possible solution is to apply multi-tier computing techniques into the wireless FL systems, which is able to enhance the stragglers' access probability to the PS and has been reported to show better learning performance compared to the conventional FL structure with direct communications between the devices and the PS [25]- [30]. In [25], the authors studied the digital two-tier relay-assisted FL framework and proposed a partially synchronized parallel scheme to simultaneously transmit the gradients from the edge devices to relays and models from relays to the PS, which is able to reduce the training time.…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, to fully exploit all the available data, one possible solution is to apply multi-tier computing techniques into the wireless FL systems, which is able to enhance the stragglers' access probability to the PS and has been reported to show better learning performance compared to the conventional FL structure with direct communications between the devices and the PS [25]- [30]. In [25], the authors studied the digital two-tier relay-assisted FL framework and proposed a partially synchronized parallel scheme to simultaneously transmit the gradients from the edge devices to relays and models from relays to the PS, which is able to reduce the training time. In [26], the authors proposed a two-tier relay-assisted AirComp-based wireless FL scheme, and optimized the transmit power coefficients at the devices and relays and the de-noising factors at the PS by minimizing the MSE of the aggregated signals with alternating minimization method.…”

Section: Introductionmentioning

confidence: 99%

“…In [30], the authors investigated the energy efficient resource allocation for a similar two-tier wireless FL systems as in [28], [29], where the devices upload the gradients to the middle-tier aggregators and the middle-tier aggregators upload the aggregated gradients immediately to the top-tier aggregator. However, none of above works considered the joint communication and learning design, i.e., analytically characterizing the impact of communications on the learning performance, and some of them focus on only the communication aspect [25]- [27] and the others only for the learning aspect [28]- [30].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Clustering and Power Control for Two-Tier Wireless Federated Learning

Guo¹,

Huang²,

Qin³

et al. 2022

Preprint

View full text Add to dashboard Cite

Federated learning (FL) has been recognized as a promising distributed learning paradigm to support intelligent applications at the wireless edge, where a global model is trained iteratively through the collaboration of the edge devices without sharing their data. However, due to the relatively large communication cost between the devices and parameter server (PS), direct computing based on the information from the devices may not be resource efficient. This paper studies the joint communication and learning design for the over-the-air computation (AirComp)-based two-tier wireless FL scheme, where the lead devices first collect the local gradients from their nearby subordinate devices, and then send the merged results to the PS for the second round of aggregation. We establish a convergence result for the proposed scheme and derive the upper bound on the optimality gap between the expected and optimal global loss values. Next, based on the device distance and data importance, we propose a hierarchical clustering method to build the two-tier structure. Then, with only the instantaneous channel state information (CSI), we formulate the optimality gap minimization problem and solve it by using an

show abstract

Partial Synchronization to Accelerate Federated Learning Over Relay-Assisted Edge Networks

Cited by 30 publications

References 25 publications

Decentralized Aggregation for Energy-Efficient Federated Learning via Overlapped Clustering and D2D Communications

Decentralized Aggregation for Energy-Efficient Federated Learning via Overlapped Clustering and D2D Communications

Decentralized Aggregation for Energy-Efficient Federated Learning via Overlapped Clustering and D2D Communications

Dynamic Clustering and Power Control for Two-Tier Wireless Federated Learning

Contact Info

Product

Resources

About