Byzantine-Robust Aggregation in Federated Learning Empowered Industrial IoT

Abstract

Federated Learning (FL) is a promising paradigm to empower on-device intelligence in Industrial Internet of Things (IIoT) due to its capability of training machine learning models across multiple IIoT devices, while preserving the privacy of their local data. However, the distributed architecture of FL relies on aggregating the parameter list from the remote devices, which poses potential security risks caused by malicious devices. In this paper, we propose a flexible and robust aggregation rule, called Auto-weighted Geometric Median (AutoGM), and analyze the robustness against outliers in the inputs. To obtain the value of AutoGM, we design an algorithm based on alternating optimization strategy. Using AutoGM as aggregation rule, we propose two robust FL solutions AutoGM_FL and AutoGM_PFL. AutoGM_FL learns a shared global model using the standard FL paradigm, and AutoGM_PFL learns a personalized model for each device. We conduct extensive experiments on the FEMNIST and Bosch IIoT datasets. The experimental results show that our solutions are robust against both model poisoning and data poisoning attacks. In particular, our solutions sustains high performance even when 30% of the nodes perform model or 50% of the nodes perform data poisoning attacks.

An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning

“…In terms of Byzantine attacks, most existing literature for distributed learning and federated learning focuses on convergence prevention [7], [19], [21], [24]. As illustrated in Fig.…”

“…In recent years, a number of Byzantine-robust techniques have been proposed [9]. They can be classified into three categories: redundancy-based schemes that assign each client redundant updates and use this redundancy to eliminate the effect of Byzantine failures [10], [11], [12], [13]; trust-based schemes that assume some of the clients or datasets are trusted for filtering and re-weighting the local model updates [14], [15], [16]; robust aggregation schemes that estimate the updates according to some robust aggregation algorithms [8], [17], [18], [19], [20], [21]. For the first category, redundancy-based schemes, in the worst case, require each node to compute Ω(M ) times more updates, where M is the number of Byzantine clients [10].…”

“…The Inner Product Manipulation (IPM) attack poses a significant threat to Median and Krum by manipulating the inner product between the true gradient and the robust aggregated gradients to be negative [24]. Other schemes, such as AutoGM [19] and Clustering [21], were proposed with only empirical evaluations.…”

An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning

“…A straightforward attack is to sample some random noise from a distribution (e.g., Gaussian distribution) and add it to the updates before uploading [19], [28]. For simplicity sake, the mean and variance of the noise are both 0.1 in our experiment.…”

“…Furthermore, Noise and IPM (ϵ = 100) eventually damage the models under the four settings by decreasing the test accuracy to around 10% (no better than random guess). This is because they both make large changes to the updates, and Mean could be easily biased by large changes [19].…”