Communication-Computation Efficient Secure Aggregation for Federated Learning

Abstract

Federated learning has been spotlighted as a way to train neural networks using data distributed over multiple nodes without the need for the nodes to share data. Unfortunately, it has also been shown that data privacy could not be fully guaranteed as adversaries may be able to extract certain information on local data from the model parameters transmitted during federated learning. A recent solution based on the secure aggregation primitive enabled privacypreserving federated learning, but at the expense of significant extra communication/computational resources. In this paper, we propose communicationcomputation efficient secure aggregation which substantially reduces the amount of communication/computational resources relative to the existing secure solution without sacrificing data privacy. The key idea behind the suggested scheme is to design the topology of the secret-sharing nodes as sparse random graphs instead of the complete graph corresponding to the existing solution. We first obtain the necessary and sufficient condition on the graph to guarantee reliable and private federated learning in the information-theoretic sense. We then suggest using the Erdős-Rényi graph in particular and provide theoretical guarantees on the reliability/privacy of the proposed scheme. Through extensive real-world experiments, we demonstrate that our scheme, using only 20 ∼ 30% of the resources required in the conventional scheme, maintains virtually the same levels of reliability and data privacy in practical federated learning systems.

Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning

“…This protocol incurs a significant communication cost due to exchanging and reconstructing the pairwise keys. Recently, several works have developed communication-efficient exchange protocols [6,7,9,14,15], which are complementary to and can be combined with our work. Another line of work focused on designing partial user selection strategies to overcome the communication bottleneck in FL while speeding up the convergence by selecting the users based on their local loss [10,11,12,13].…”

SAFE: Secure Aggregation with Failover and Encryption

“…A secure aggregation protocol for Federated Learning was first outlined in [2] with the full details published in [4] (referred to as the BON protocol below), and then extended in [6] and [5].…”

“…In [6] the authors also recognize the scalability and computational complexity of the BON protocol and improve on it by splitting up the aggregation across subgraphs. Such subgraph splitting could also be applied on top of our protocol, and our subgroup feature does exactly that.…”

“…In [5] the authors also propose a topology overlay across the BON primitives, called Turbo-Aggregate, and in particular a circular subgroup one as opposed to a sparse subgraph like in [6]. Although our approach also relies on a circular topology and subgrouping, our masking primitives are distinctly different from the ones in BON not to suffer from the same scalablity issues or computational complexity (quadratic in number of users), and thus subgrouping is only required in our case when there is a large number of users.…”

“…The protocol proposed in [4] and variations thereof [5,6] target learners connected to controllers by unreliable wireless links, but have the following shortcomings:…”