Asynchronous Federated Learning Over Wireless Communication Networks

Abstract

In this paper, federated learning (FL) over wireless networks is investigated. In each communication round, a subset of devices is selected to participate in the aggregation with limited time and energy. In order to minimize the convergence time, global loss and latency are jointly considered in a Stackelberg game based framework. Specifically, age of information (AoI) based device selection is considered at leader-level as a global loss minimization problem, while sub-channel assignment, computational resource allocation, and power allocation are considered at follower-level as a latency minimization problem. By dividing the follower-level problem into two sub-problems, the best response of the follower is obtained by a monotonic optimization based resource allocation algorithm and a matching based subchannel assignment algorithm. By deriving the upper bound of convergence rate, the leader-level problem is reformulated, and then a list based device selection algorithm is proposed to achieve Stackelberg equilibrium. Simulation results indicate that the proposed device selection scheme outperforms other schemes in terms of the global loss, and the developed algorithms can significantly decrease the time consumption of computation and communication.

Time-triggered Federated Learning over Wireless Networks

“…Applying (23) to the second term of (44), and (22) to the last term of (44), then according to (19), we have…”

“…Despite its benefits in high communication efficiency, Sync FL can result in low training efficiency, since its convergence speed is limited by the slowest users, known as stragglers [22]. To handle the straggler issue, Async FL is a more flexible solution by allowing Async aggregation without waiting for the stragglers [23,24]. However, Async FL has its own unique challenges, including: 1) the staleness problem, where the local model based on an old global model may be harmful to the current aggregation.…”

A Topology-based Approach for Uplink Transmission Optimization in Wireless Federated Learning

“…Some other notable studies have concentrated on improving the effectiveness of the training process itself. Examples include modifying the fundamental framework of FL, such as a novel asynchronous FL framework that adapts well to the heterogeneity of communication environments [19], an efficient communication FL approach that provides users with a training strategy to achieve fast convergence [20], introducing binary neural networks to replace the conventional neural networks with real values to meet the strict latency and efficiency constraints at the edge of wireless networks [21], and investigating FL over a multihop wireless network with in-network model aggregation [22].…”

“…Practically, this would also help to avoid communication overhead due to frequent reallocation of the frequency spectrum. We will consider P3: Minimize 10), ( 11), ( 12), ( 13), ( 14), (19).…”

Semi-Asynchronous Hierarchical Federated Learning Over Mobile Edge Networks

“…Meanwhile, when one device uploads its model, the others continue to complete their training. The authors in [22] proposed a novel asynchronous FL mechanism to coordinate the heterogeneity of devices, communication environments, and learning tasks. Nevertheless, the training round under asynchronous methods is higher than synchronous methods.…”