DeSMP: Differential Privacy-exploited Stealthy Model Poisoning Attacks in Federated Learning

Abstract

Federated learning (FL) has become an emerging machine learning technique lately due to its efficacy in safeguarding the client's confidential information. Nevertheless, despite the inherent and additional privacy-preserving mechanisms (e.g., differential privacy, secure multi-party computation, etc.), the FL models are still vulnerable to various privacy-violating and security-compromising attacks (e.g., data or model poisoning) due to their numerous attack vectors which in turn, make the models either ineffective or suboptimal. Existing adversarial models focusing on untargeted model poisoning attacks are not enough stealthy and persistent at the same time because of their conflicting nature (large scale attacks are easier to detect and vice versa) and thus, remain an unsolved research problem in this adversarial learning paradigm. Considering this, in this paper, we analyze this adversarial learning process in an FL setting and show that a stealthy and persistent model poisoning attack can be conducted exploiting the differential noise. More specifically, we develop an unprecedented DP-exploited stealthy model poisoning (DeSMP) attack for FL models. Our empirical analysis on both the classification and regression tasks using two popular datasets reflects the effectiveness of the proposed DeSMP attack. Moreover, we develop a novel reinforcement learning (RL)based defense strategy against such model poisoning attacks which can intelligently and dynamically select the privacy level of the FL models to minimize the DeSMP attack surface and facilitate the attack detection.

Adversarial Analysis of the Differentially-Private Federated Learning in Cyber-Physical Critical Infrastructures

“…Trimmed mean [28], Median [28], etc.) or ( 3) state-of-theart DPFL aggregation methods [12], [15]. Particularly, we focus on the aggregation scheme of [12] which proposes to add a bounded DP-noise to the sum of the all clipped local updates, ∆w n to form a global model, ∆w g .…”

“…For a comprehensive understanding of recently discovered DP-exploited poisoning attacks [13], [14], [15], [29], we can consider Fig. 3 where multiple edge nodes are collaboratively updating the global model.…”

“…However, recent research on the privacy and security challenges in the CPCI domain points out the vulnerability and the exploitation opportunity of the DP-mechanism to conduct devastating poisoning attacks [13], [14]. Particularly, in FL-based CPCIs [4], [5], [6], where the model is trained on a diffuse network of edge devices using local data, DP can introduce new attack vectors for conducting stealthy poisoning attacks [15], [16]. Therefore, although DP can effectively act against certain privacy-violating attacks (e.g., membership inference attacks (MIAs)) in CPCIs [12], [14], it can be exploited to falsify the sensor data (data poisoning) or manipulate the local model parameters (model poisoning) through stealthy poisoning.…”

On the Security & Privacy in Federated Learning

“…However, defenses have arisen towards this Boosting-based model replacement attacks because it increases the l n norm of the vector of weights (see Section VI). Additionally, several proposed stealthy techniques evaded such defenses [48], [49], [50], [51], [52], [53]. Furthermore, [54] proposed Sybils that evaded and improved attacks quality.…”