DReS-FL: Dropout-Resilient Secure Federated Learning for Non-IID Clients via Secret Data Sharing

Abstract

Federated learning (FL) strives to enable privacy-preserving training of machine learning models without centrally collecting clients' private data. Despite its advantages, the local datasets across clients in FL are non-independent and identically distributed (non-IID), and the data-owning clients may drop out of the training process arbitrarily. These characteristics will significantly degrade the training performance. Therefore, we propose a Dropout-Resilient Secure Federated Learning (DReS-FL) framework based on Lagrange coded computing (LCC) to tackle both the non-IID and dropout problems. The key idea is to utilize Lagrange coding to secretly share the private datasets among clients so that each client receives an encoded version of the global dataset 1 , and the local gradient computation over this dataset is unbiased. To correctly decode the gradient at the server, the gradient function has to be a polynomial in a finite field, and thus we construct polynomial integer neural networks (PINNs) to enable our framework. Theoretical analysis shows that DReS-FL is resilient to client dropouts and provides strong privacy guarantees. Furthermore, we experimentally demonstrate that DReS-FL consistently leads to significant performance gains over baseline methods.

Optimization-Based Block Coordinate Gradient Coding for Mitigating Partial Stragglers in Distributed Learning

“…As one of the important frameworks to distributedly train machine learning model, federated learning has also been combined with coding techniques in recent studies, such as CodedFedL [30], CodedPaddedFL [31] and DRes-FL [32].…”