Semi-Decentralized Federated Edge Learning for Fast Convergence on Non-IID Data

Sun, Yuchang; Shao, Jiawei; Mao, Yuyi; Wang, Jessie Hui; Zhang, Jun

doi:10.1109/wcnc51071.2022.9771904

Cited by 27 publications

(14 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where C i,m comm = k∈Cm C i,m,k comm is the communication cost of edge node m at time slot i. Following [26], the communication cost between edge m and client k at time slot i is calculated by…”

Section: Adaptive Staleness Control At Edge-cloud Layermentioning

confidence: 99%

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Wu¹,

Chen²,

Ouyang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a promising paradigm that enables collaboratively learning a shared model across massive clients while keeping the training data locally. However, for many existing FL systems, clients need to frequently exchange model parameters of large data size with the remote cloud server directly via wide-area networks (WAN), leading to significant communication overhead and long transmission time. To mitigate the communication bottleneck, we resort to the hierarchical federated learning paradigm of HiFL, which reaps the benefits of mobile edge computing and combines synchronous client-edge model aggregation and asynchronous edge-cloud model aggregation together to greatly reduce the traffic volumes of WAN transmissions. Specifically, we first analyze the convergence bound of HiFL theoretically and identify the key controllable factors for model performance improvement. We then advocate an enhanced design of HiFlash by innovatively integrating deep reinforcement learning based adaptive staleness control and heterogeneity-aware client-edge association strategy to boost the system efficiency and mitigate the staleness effect without compromising model accuracy. Extensive experiments corroborate the superior performance of HiFlash in model accuracy, communication reduction, and system efficiency.

show abstract

Section: Adaptive Staleness Control At Edge-cloud Layermentioning

confidence: 99%

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Wu¹,

Chen²,

Ouyang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, due to the requirement of global model aggregation with two-time scale FL over both D2D and user-to-CPS wireless transmission, this FL scheme has limited latency improvement. In [25], [26], the authors developed FL model dissemination schemes by leveraging connected edge servers (ESs), which aggregate local models from their UD clusters and exchange them with all the other ESs in the network for global aggregation. However, a fully connected ES network is prohibitively expensive in practice, especially when ESs are connected by wireless links.…”

Section: A Summary Of the Related Workmentioning

confidence: 99%

“…However, a fully connected ES network is prohibitively expensive in practice, especially when ESs are connected by wireless links. In addition, each global iteration of FL framework takes significantly longer because ESs continue to transmit local aggregated models until all other ESs receive them successfully [25], [26]. The authors in [27] addressed this issue by introducing conflicting UDs, which are the UDs covering multiple clusters, and allowing parameter exchanges between them and local model aggregators.…”

Section: A Summary Of the Related Workmentioning

confidence: 99%

“…Additionally, mmWave band requires LOS links between UDs and UAVs for local model aggregation, as well as LOS UAV-to-UAV links for model dissemination. Accordingly, the FL model dissemination frameworks proposed in [25]- [27] will not be applicable to mmWave ATINs. We emphasize that in order to maintain convergence speed and reduce energy consumption, the interaction among UD-to-UAV associations, RRB scheduling, and UAV-to-UAV link selection, in addition to the inherent properties of mmWave bands, must be appropriately characterized.…”

Section: A Summary Of the Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Decentralized Aggregation for Energy-Efficient Federated Learning via D2D Communications

Al-Abiad

Obeed

Hossain

et al. 2023

IEEE Trans. Commun.

View full text Add to dashboard Cite

It is anticipated that aerial-terrestrial integrated networks incorporating unmanned aerial vehicles (UAVs) mounted relays will offer improved coverage and connectivity in the beyond 5G era. Meanwhile, federated learning (FL) is a promising distributed machine learning technique for building inference models over wireless networks due to its ability to maintain user privacy and reduce communication overhead. However, off-the-shelf FL models aggregate global parameters at a central parameter server (CPS), increasing energy consumption and latency, as well as inefficiently utilizing radio resource blocks (RRBs) for distributed user devices (UDs). This paper presents a resource-efficient FL framework, called FedMoD (federated learning with model dissemination), for millimeterwave (mmWave) aerial-terrestrial integrated networks with the following two unique characteristics. Firstly, FedMoD presents a novel decentralized model dissemination algorithm that makes use of UAVs as local model aggregators through UAV-to-UAV and device-to-device (D2D) communications. As a result, FedMoD (i) increases the number of participant UDs in developing FL model and (ii) achieves global model aggregation without involving CPS. Secondly, FedMoD reduces the energy consumption of FL using radio resource management (RRM) under the constraints of over-the-air learning latency. In order to achieve this, by leveraging graph theory, FedMoD optimizes the scheduling of line-of-sight (LOS) UDs to suitable UAVs/RRBs over mmWave links and non-LOS UDs to available LOS UDs via overlay D2D communications. Extensive simulations reveal that decentralized FedMoD offers same convergence rate performance as compared to conventional FL frameworks.

show abstract

“…Client clustering and resource allocations for inter-cluster communications are two key technical issues investigated for the DDFL framework. Another example of this type of hybrid architecture is the SD-FEEL framework [118], which comprises multiple FL servers deployed on edge nodes (called edge servers). Each edge server coordinates a cluster of client nodes for local model updating and intra-cluster model aggregation.…”

Section: Hybrid Hierarchical and Peer-to-peer Fl Architecturementioning

confidence: 99%

Combining Federated Learning and Edge Computing toward Ubiquitous Intelligence: Challenges, Recent Advances, and Future Directions

Duan¹,

Huang²,

Hu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Full leverage of the huge volume of data generated on a large number of user devices for providing intelligent services in the 6G network calls for Ubiquitous Intelligence (UI). A key to developing UI lies in the involvement of the large number of network devices, which contribute their data to collaborative Machine Learning (ML) and provide their computational resources to support the learning process. Federated Learning (FL) is a new ML method that enables data owners to collaborate in model training without exposing private data, which allows user devices to contribute their data to developing UI. Edge computing deploys cloud-like capabilities at the network edge, which enables network devices to offer their computational resources for supporting FL. Therefore, a combination of FL and edge computing may greatly facilitate the development of ubiquitous intelligence. In this article, we present a comprehensive survey of the recent developments in technologies for combining FL and edge computing with a holistic vision across the fields of FL and edge computing. We conduct our survey from both the perspective of an FL framework deployed in an edge computing environment (FL in Edge) and the perspective of an edge computing system providing a platform for supporting FL (Edge for FL). From the FL in Edge perspective, we first identify the main challenges to FL in edge computing and then survey the representative technical strategies for addressing the challenges. From the Edge for FL perspective, we first analyze the key requirements for edge computing to support FL and then review the recent advances in edge computing technologies that may be exploited to meet the requirements. Then we discuss open problems and identify some possible directions for future research on combining FL and edge computing, with the hope of arousing the research community’s interest in this emerging and exciting interdisciplinary field.

show abstract

Semi-Decentralized Federated Edge Learning for Fast Convergence on Non-IID Data

Cited by 27 publications

References 17 publications

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

Decentralized Aggregation for Energy-Efficient Federated Learning via D2D Communications

Combining Federated Learning and Edge Computing toward Ubiquitous Intelligence: Challenges, Recent Advances, and Future Directions

Contact Info

Product

Resources

About