FLEE: A Hierarchical Federated Learning Framework for Distributed Deep Neural Network over Cloud, Edge, and End Device

Zhong, Zhengyi; Bao, Weidong; Wang, Ji; Zhu, Xiaomin; Zhang, Xiongtao

doi:10.1145/3514501

Cited by 10 publications

(4 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Specifically, the cloud layer coordinates the edge layer, while the edge layer cooperates with the terminal layer. Since the edge layer has taken on the communication overhead from the cloud layer and the computation from the terminal layer [171], this architecture has been shown the capability of high communication efficiency and low latency in the asynchronous and heterogeneous system [146,168]. For further improving the system robustness, wireless channel has been considered for hierarchical over-the-air FL (HOTAFL) [12].…”

Section: Centralized Flmentioning

confidence: 99%

Advancements in Federated Learning: Models, Methods, and Privacy

Chen¹,

Wang²,

Jin³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated learning (FL) is a promising technique for resolving the rising privacy and security concerns. Its main ingredient is to cooperatively learn the model among the distributed clients without uploading any sensitive data. In this paper, we conducted a thorough review of the related works, following the development context and deeply mining the key technologies behind FL from the perspectives of theory and application. Specifically, we first classify the existing works in FL architecture based on the network topology of FL systems with detailed analysis and summarization. Next, we abstract the current application problems, summarize the general techniques and frame the application problems into the general paradigm of FL base models. Moreover, we provide our proposed solutions for model training via FL. We have summarized and analyzed the existing FedOpt algorithms, and deeply revealed the algorithmic development principles of many first-order algorithms in depth, proposing a more generalized algorithm design framework. With the instantiation of these frameworks, FedOpt algorithms can be simply developed. As privacy and security is the fundamental requirement in FL, we provide the existing attack scenarios and the defense methods. To the best of our knowledge, we are among the first tier to review the theoretical methodology and propose our strategies since there are very few works surveying the theoretical approaches. Our survey targets motivating the development of high-performance, privacy-preserving, and secure methods to integrate FL into real-world applications.CCS Concepts: • Theory of computation → Theory and algorithms for application domains; • Networks → Network architectures; • Computing methodologies → Distributed computing methodologies; • Security and privacy;

show abstract

Section: Centralized Flmentioning

confidence: 99%

Advancements in Federated Learning: Models, Methods, and Privacy

Chen¹,

Wang²,

Jin³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Multiple works have formulated client association and resource allocation problems to jointly optimize computation and communication efficiency in synchronous hierarchical FL [6,7,41,42]. Recent efforts studied mobility-aware [21] and dynamic hierarchical aggregations for new data [62]. SHARE [19] separated the device selection and device-gateway association into two subproblems, then jointly minimized communication cost and shaped data distribution at aggregators for better global accuracy.…”

Section: Related Workmentioning

confidence: 99%

“…How to preserve the positive gains while avoiding undesired degradation during scaling to hierarchical architectures remains an active research topic. While previous works have studied how to improve FL convergence under one or two of data heterogeneity [31,39,50], system heterogeneity [13,34,38], unexpected stragglers [44], and hierarchical FL for better scalability [21,62], none of existing work provides a systematic solution to address all challenges in a hierarchical and unreliable IoT network. Our work is the first end-to-end framework that uses (i) asynchronous and hierarchical FL algorithm and (ii) system management design to enhance efficiency and robustness, for handling all challenges (C1)-(C4).…”

Section: Introductionmentioning

confidence: 99%

Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks

Yu¹,

Cherkasova²,

Vardhan³

et al. 2023

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) has gained increasing interest in recent years as a distributed on-device learning paradigm. However, multiple challenges remain to be addressed for deploying FL in real-world Internet-of-Things (IoT) networks with hierarchies. Although existing works have proposed various approaches to account data heterogeneity, system heterogeneity, unexpected stragglers and scalibility, none of them provides a systematic solution to address all of the challenges in a hierarchical and unreliable IoT network. In this paper, we propose an asynchronous and hierarchical framework (Async-HFL) for performing FL in a common three-tier IoT network architecture. In response to the largely varied delays, Async-HFL employs asynchronous aggregations at both the gateway and the cloud levels thus avoids long waiting time. To fully unleash the potential of Async-HFL in converging speed under system heterogeneities and stragglers, we design device selection at the gateway level and device-gateway association at the cloud level. Device selection chooses edge devices to trigger local training in real-time while device-gateway association determines the network topology periodically after several cloud epochs, both satisfying bandwidth limitation. We evaluate Async-HFL's convergence speedup using large-scale simulations based on ns-3 and a network topology from NYCMesh. Our results show that Async-HFL converges 1.08-1.31x faster in wall-clock time and saves up to 21.6% total communication cost compared to state-of-the-art asynchronous FL algorithms (with client selection). We further validate Async-HFL on a physical deployment and observe robust convergence under unexpected stragglers.

show abstract

“…O ATHENA-FL agrupa os clientes conforme as distribuic ¸ões dos dados, antes de realizar o treinamento do modelo OvA e, dessa forma, reduz o tempo de treinamento dos detectores. FLEE (Federated Learning Early Exit of inference) é um arcabouc ¸o de aprendizado federado hierárquico que divide o modelo em três localizac ¸ões diferentes [Zhong et al 2022]. A divisão do modelo entre a nuvem, borda e dispositivo final permite utilizar o método de saídas antecipadas de rede neural no processo de inferência.…”

Section: Personalizac ¸ãO De Modelos No Aprendizado Federadounclassified

ATHENA-FL: Evitando a Heterogeneidade Estatística através do Um-contra-Todos no Aprendizado Federado

Souza

Camilo

Rebello

et al. 2023

Anais Do VII Workshop De Computação Urbana (CoUrb 2023)

View full text Add to dashboard Cite

O aprendizado federado é um novo paradigma que permite o treinamento de modelos de aprendizado de máquina através da colaboração entre clientes e um servidor de agregação. O treinamento dispensa o compartilhamento de dados privados, garantindo aos clientes privacidade de suas amostras. Entretanto, quando os clientes possuem distribuições de dados distintas, o treinamento apresenta dificuldades de convergência, resultando em erros preditivos no modelo final. Este artigo propõe um sistema de aprendizado federado que considera clientes com distribuições de dados heterogêneas e, mesmo assim, produz modelos acurados em menos épocas de treinamento do que o estado da arte. Os efeitos da heterogeneidade dos dados são mitigados através do agrupamento dos clientes baseado em uma estimativa da distribuição de dados através dos pesos da rede neural treinada localmente. Além disso, o sistema utiliza a técnica um-contra-todos, treina um detector para cada classe no sistema. Assim, grupos diferentes podem combinar os detectores a fim de formar um modelo capaz de detectar classes provenientes de outros grupos. Os resultados mostram que o modelo um-contra-todos possui alta capacidade de identificar corretamente as amostras e com acurácia até 18% maior do que o treinamento tradicional, com um baixo custo de comunicação durante o treinamento, reduzindo a quantidade de bytes transmitidos entre 59,6% até 94% em comparação à arquitetura MobileNet.

show abstract

FLEE: A Hierarchical Federated Learning Framework for Distributed Deep Neural Network over Cloud, Edge, and End Device

Cited by 10 publications

References 45 publications

Advancements in Federated Learning: Models, Methods, and Privacy

Advancements in Federated Learning: Models, Methods, and Privacy

Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks

ATHENA-FL: Evitando a Heterogeneidade Estatística através do Um-contra-Todos no Aprendizado Federado

Contact Info

Product

Resources

About