Balancing Privacy and Performance: A Differential Privacy Approach in Federated Learning

Abstract

Federated learning (FL), a decentralized approach to machine learning, facilitates model training across multiple devices, ensuring data privacy. However, achieving a delicate privacy preservation–model convergence balance remains a major problem. Understanding how different hyperparameters affect this balance is crucial for optimizing FL systems. This article examines the impact of various hyperparameters, like the privacy budget (ϵ), clipping norm (C), and the number of randomly chosen clients (K) per communication round. Through a comprehensive set of experiments, we compare training scenarios under both independent and identically distributed (IID) and non-independent and identically distributed (Non-IID) data settings. Our findings reveal that the combination of ϵ and C significantly influences the global noise variance, affecting the model’s performance in both IID and Non-IID scenarios. Stricter privacy conditions lead to fluctuating non-converging loss behavior, particularly in Non-IID settings. We consider the number of clients (K) and its impact on the loss fluctuations and the convergence improvement, particularly under strict privacy measures. Thus, Non-IID settings are more responsive to stricter privacy regulations; yet, with a higher client interaction volume, they also can offer better convergence. Collectively, knowledge of the privacy-preserving approach in FL has been extended and useful suggestions towards an ideal privacy–convergence balance were achieved.