THF: 3-Way Hierarchical Framework for Efficient Client Selection and Resource Management in Federated Learning

Abstract

Federated Learning (FL) is a promising technique for collaboratively training machine learning models on massively distributed clients data under privacy constraints. However, the existing FL literature focuses on speeding-up the learning process and ignores minimizing the communication cost which is critical for resource-constrained clients. To this end, in this paper, we propose a novel 3-way hierarchical framework (THF) to promote communication efficiency in FL. Using the proposed framework, only a cluster-head (CH) communicates with the cloud server through edge-aggregation in order to minimize the communication cost of clients. In particular, the clients upload their local models to their respective CHs, which are responsible to forward them to the corresponding edge-server. The edgeserver averages the local models and iterates until it achieves the edge-accuracy. Afterward, each edge-server uploads the edgemodels to the cloud server for global aggregation. In this way, model downloading and uploading requires less bandwidth due to the short distance from source to destination that makes an efficient 3-way hierarchical network structure. In addition, we formulate a joint communication and computation resource management scheme through efficient client selection in order to achieve global cost minimization in FL. We conduct extensive empirical evaluations on diverse data learning tasks on multiple datasets to signify that THF achieves global cost savings and converges within fewer communication rounds compared to other FL approaches.

Communication Efficiency and Non-Independent and Identically Distributed Data Challenge in Federated Learning: A Systematic Mapping Study

“…The client selection, select model updates, over-the-air computation, clustering, periodic model averaging, asynchronous, and two-level aggregation techniques aim to reduce the number of updates between the server and the clients. The client selection scheme is used to select the clients that can contribute more to enhance the global model, which results in a reduction in the communication rounds [147,185,191]. Using asynchronous communication can enhance the global model performance by allowing the aggregation of the received model without waiting for all clients [146,171,211].…”

A Comprehensive Overview of IoT-Based Federated Learning: Focusing on Client Selection Methods

“…The main issue in data heterogeneity in clustering is non-IID data issues [ 10 , 13 , 28 ]. The clustering method can be based on training data [ 31 , 89 , 90 ] or based on the location of clients and the required skills and efficient collaboration among each other [ 2 , 13 ]. Some work performed clustering if necessary [ 28 ] and handled varying client populations.…”

“…ML algorithms can effectively utilize this data to derive valuable insights, enable real-time decision-making, and enhance process optimization. In ML used in conjunction with the IoT, there is a concern about the amount of data involved in the training process, especially when the data are sensitive [ 1 , 2 , 3 ]. One of the most promising solutions to the isolated data island [ 1 ] problem is FL, where many clients ranging from edge devices to IoT devices collaboratively train a model under the orchestration of a central server.…”

“…The increasing number and type of network services and the proliferation of mobile edge have prompted the deployment of IoT [ 2 ] devices with advanced sensors, computing, and communication capabilities for crowd-sensing tasks [ 42 ]. The advent of AI has led to significant developments in numerous modern applications, such as air quality, weather monitoring, and video surveillance [ 1 , 2 ].…”

“…The increasing number and type of network services and the proliferation of mobile edge have prompted the deployment of IoT [ 2 ] devices with advanced sensors, computing, and communication capabilities for crowd-sensing tasks [ 42 ]. The advent of AI has led to significant developments in numerous modern applications, such as air quality, weather monitoring, and video surveillance [ 1 , 2 ]. Nowadays, ML algorithms and intelligent applications have made it possible to analyze various types of data, including text, numeric, photographs, videos, and locations, from different IoT devices [ 20 , 43 , 44 ].…”