Cryptonite: A Framework for Flexible Time-Series Secure Aggregation with Non-interactive Fault Recovery

Karl, Ryan; Takeshita, Jonathan; Jung, Taeho

doi:10.1007/978-3-030-90019-9_16

Cited by 3 publications

(4 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Motivated by these challenges, we propose a novel incremental Asynchronous Secure Aggregation algorithm that uses a Trusted Execution Environment (Karl et al, 2020b) in Section 5.…”

Section: System Design Challenges In Asyncflmentioning

confidence: 99%

See 1 more Smart Citation

Papaya: Practical, Private, and Scalable Federated Learning

Huba¹,

Nguyen²,

Malik³

et al. 2021

Preprint

View full text Add to dashboard Cite

Cross-device Federated Learning (FL) is a distributed learning paradigm with several challenges that differentiate it from traditional distributed learning, variability in the system characteristics on each device, and millions of clients coordinating with a central server being primary ones. Most FL systems described in the literature are synchronous -they perform a synchronized aggregation of model updates from individual clients. Scaling synchronous FL is challenging since increasing the number of clients training in parallel leads to diminishing returns in training speed, analogous to large-batch training. Moreover, stragglers hinder synchronous FL training. In this work, we outline a production asynchronous FL system design. Our work tackles the aforementioned issues, sketches of some of the system design challenges and their solutions, and touches upon principles that emerged from building a production FL system for millions of clients. Empirically, we demonstrate that asynchronous FL converges faster than synchronous FL when training across nearly one hundred million devices. In particular, in high concurrency settings, asynchronous FL is 5× faster and has nearly 8× less communication overhead than synchronous FL.

show abstract

“…Motivated by these challenges, we propose a novel incremental Asynchronous Secure Aggregation algorithm that uses a Trusted Execution Environment (Karl et al, 2020b) in Section 5.…”

Section: System Design Challenges In Asyncflmentioning

confidence: 99%

“…In this section we will go over the detailed design of our protocol and formally prove its security. We adopt the same strategy from Cryptonite (Karl et al, 2020a). A trusted party realized by trusted hardware (e.g., Intel SGX) will assist with the procedure and help make up for the dropped clients.…”

Section: B Protocol Design and Security Proofmentioning

confidence: 99%

Papaya: Practical, Private, and Scalable Federated Learning

Huba¹,

Nguyen²,

Malik³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Aggregation [5][6][7] ensures that individual client updates are aggregated together before they are visible to the server. For differential privacy (DP), the DP-SGD algorithm [8] can be used to provide user-level DP guarantees in FL [9].…”

Section: Introductionmentioning

confidence: 99%

“…1000) require less noise with DP, at the cost of slower training convergence. Unlike other asynchronous FL proposals [18], FedBuff is compatible with Secure Aggregation, especially techniques that rely on a TEE [6,7].…”

Section: Introductionmentioning

confidence: 99%

Federated Learning with Buffered Asynchronous Aggregation

Nguyen¹,

Malik²,

Zhan³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated Learning (FL) trains a shared model across distributed devices while keeping the training data on the devices. Most FL schemes are synchronous: they perform a synchronized aggregation of model updates from individual devices. Synchronous training can be slow because of late-arriving devices (stragglers). On the other hand, completely asynchronous training makes FL less private because of incompatibility with secure aggregation. In this work, we propose a model aggregation scheme, FedBuff, that combines the best properties of synchronous and asynchronous FL. Similar to synchronous FL, FedBuff is compatible with secure aggregation. Similar to asynchronous FL, FedBuff is robust to stragglers. In FedBuff, clients trains asynchronously and send updates to the server. The server aggregates client updates in a private buffer until K updates have been received, at which point a server model update is immediately performed. We provide theoretical convergence guarantees for FedBuff in a non-convex setting. Empirically, FedBuff converges up to 3.8× faster than previous proposals for synchronous FL (e.g., FedAvgM), and up to 2.5× faster than previous proposals for asynchronous FL (e.g., FedAsync). We show that FedBuff is robust to different staleness distributions and is more scalable than synchronous FL techniques.Preprint. Under review.

show abstract