Decentralized Edge Intelligence: A Dynamic Resource Allocation Framework for Hierarchical Federated Learning

Lim, Wei Yang Bryan; Ng, Jer Shyuan; Xiong, Zehui; Jin, Jiangming; Zhang, Yang; Niyato, Dusit; Leung, Cyril; Chen, Miao

doi:10.1109/tpds.2021.3096076

Cited by 192 publications

(51 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, FL reduces communication costs since only the model parameters are transmitted to the parameter server rather than the raw data. This is important, especially when the training data is 3D objects or high-resolution video streams [225]. Second, FL models can be constantly trained locally to allow continual learning.…”

Section: Privacy and Securitymentioning

confidence: 99%

A Full Dive into Realizing the Edge-enabled Metaverse: Visions, Enabling Technologies,and Challenges

Xu¹,

Ng²,

Lim³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Dubbed "the successor to the mobile Internet", the concept of the Metaverse has grown in popularity. While there exist lite versions of the Metaverse today, they are still far from realizing the full vision of an immersive, embodied, and interoperable Metaverse. Without addressing the issues of implementation from the communication and networking, as well as computation perspectives, the Metaverse is difficult to succeed the Internet, especially in terms of its accessibility to billions of users today. In this survey, we focus on the edge-enabled Metaverse to realize its ultimate vision. We first provide readers with a succinct tutorial of the Metaverse, an introduction to the architecture, as well as current developments. To enable the ubiquitous, seamless, and embodied access to the Metaverse, we discuss the communication and networking challenges and survey cutting-edge solutions and concepts that leverage next-generation communication systems for users to be telepresent and teleoperate in the Metaverse. Moreover, given the high computation costs required, e.g., to render immersive 3D worlds and run data hungry artificial intelligence (AI) driven applications, we discuss the computation challenges and cloud-edge-end computation framework driven solutions to realize the Metaverse on resource-constrained edge devices. Next, we explore how blockchain technologies can aid in the interoperable development of the Metaverse, not just in terms of empowering the economic circulation of virtual usergenerated contents, but also to manage physical edge resources in a decentralized, transparent, and tamper-proof manner. Finally, we discuss the future research directions towards realizing the true vision of the edge-enabled Metaverse.

show abstract

Section: Privacy and Securitymentioning

confidence: 99%

A Full Dive into Realizing the Edge-enabled Metaverse: Visions, Enabling Technologies,and Challenges

Xu¹,

Ng²,

Lim³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…An alternative approach to both centralized and decentralized schemes is the hierarchical FL (HFL) framework [27,[40][41][42], in which multiple PSs are employed for the aggregation to prevent a communication bottleneck. In HFL, clients are divided into clusters and a PS is assigned to each cluster to perform local aggregation, while the aggregated models at the clusters are later re-aggregated at the main PS in a subsequent step to obtain the global model.…”

Section: Related Workmentioning

confidence: 99%

Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying

Yemini¹,

Saha²,

Ozfatura³

et al. 2022

Preprint

View full text Add to dashboard Cite

Intermittent client connectivity is one of the major challenges in centralized federated edge learning frameworks. Intermittently failing uplinks to the central parameter server (PS) can induce a large generalization gap in performance especially when the data distribution among the clients exhibits heterogeneity. In this work, to mitigate communication blockages between clients and the central PS, we introduce the concept of knowledge relaying wherein the successfully participating clients collaborate in relaying their neighbors' local updates to a central parameter server (PS) in order to boost the participation of clients with intermittently failing connectivity. We propose a collaborative relaying based semi-decentralized federated edge learning framework where at every communication round each client first computes a local consensus of the updates from its neighboring clients and eventually transmits a weighted average of its own update and those of its neighbors to the PS. We appropriately optimize these averaging weights to reduce the variance of the global update at the PS while ensuring that the global update is unbiased, consequently improving the convergence rate. Finally, by conducting experiments on CIFAR-10 dataset we validate our theoretical results and demonstrate that our proposed scheme is superior to Federated averaging benchmark especially when data distribution among clients is non-iid.* Michal Yemini and Andrea J. Goldsmith are with the Faculty of Electrical and Computer Engineering, Princeton University.

show abstract

“…For deploying FL at the network edge, substantial efforts haven been made on resource allocation [4], [5], [9], [10], transmission scheduling [6]- [8], and learning algorithm refinement [11]- [15]. In [4], the problem of resource allocation was investigated in the proposed hierarchical FL framework with devices clustered to train models before reaching to the global aggregator.…”

Section: Related Workmentioning

confidence: 99%

“…An energy-efficient radio resource allocation scheme devised in [5] assigned more bandwidth to FEL participants with lower computing power for the sake of aggregation synchronization. Regarding a new paradigm named hierarchical federated learning using the intermediate model aggregation to achieve higher communication efficiency, Lim et al [9], [10] studied the dynamic resource allocation with the help of game theoretical tools, including the evolutionary game, Stackelberg game, and auction. Yang et al [6] studied three classes of transmission scheduling mechanisms, i.e., random, round-robin, and proportional fair, aiming at the optimal convergence rate for FL.…”

Section: Related Workmentioning

confidence: 99%

Solving the Federated Edge Learning Participation Dilemma: A Truthful and Correlated Perspective

Zou

et al. 2022

IEEE Trans. Veh. Technol.

View full text Add to dashboard Cite

An emerging computational paradigm, named federated edge learning (FEL), enables intelligent computing at the network edge with the feature of preserving data privacy for edge devices. Given their constrained resources, it becomes a great challenge to achieve high execution performance for FEL. Most of the state-of-the-arts concentrate on enhancing FEL from the perspective of system operation procedures, taking few precautions during the composition step of the FEL system. Though a few recent studies recognize the importance of FEL formation and propose server-centric device selection schemes, the impact of data sizes is largely overlooked. In this paper, we take advantage of game theory to depict the decision dilemma among edge devices regarding whether to participate in FEL or not given their heterogeneous sizes of local datasets. For realizing both the individual and global optimization, the server is employed to solve the participation dilemma, which requires accurate information collection for devices' local datasets. Hence, we utilize mechanism design to enable truthful information solicitation. With the help of correlated equilibrium, we derive a decision making strategy for devices from the global perspective, which can achieve the longterm stability and efficacy of FEL. For scalability consideration, we optimize the computational complexity of the basic solution to the polynomial level. Lastly, extensive experiments based on both real and synthetic data are conducted to evaluate our proposed mechanisms, with experimental results demonstrating the performance advantages.

show abstract

Decentralized Edge Intelligence: A Dynamic Resource Allocation Framework for Hierarchical Federated Learning

Abstract: Decentralized edge intelligence : a dynamic resource allocation framework for hierarchical federated learning. IEEE Transactions On Parallel and Distributed Systems, 33(3), 536-550.

Cited by 192 publications

References 36 publications

A Full Dive into Realizing the Edge-enabled Metaverse: Visions, Enabling Technologies,and Challenges

A Full Dive into Realizing the Edge-enabled Metaverse: Visions, Enabling Technologies,and Challenges

Robust Federated Learning with Connectivity Failures: A Semi-Decentralized Framework with Collaborative Relaying

Solving the Federated Edge Learning Participation Dilemma: A Truthful and Correlated Perspective

Contact Info

Product

Resources

About