MYSTIKO: Cloud-Mediated, Private, Federated Gradient Descent

Jayaram, K. R.; Verma, Archit; Verma, Ashish; Thomas, Gegi; Sutcher-Shepard, Colin

doi:10.1109/cloud49709.2020.00039

Cited by 9 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is safe to assume that 2 + 2 + 2 ≥ 1. Thus, substituting from ( 16) in (18), we obtain that, for each ∈ {1, . .…”

Section: A Impracticality Results and Proofsmentioning

confidence: 99%

“…The idea mainly consists in adding DP noise to the gradients computed by the different workers, as explained in Section 2.3. Other techniques have investigated encrypting the gradients shared in the distributed network [18,35] to prevent a passive attacker from violating the privacy of data nodes by simply intercepting the gradients exchanged. However, the Byzantine resilience aspect of distributed SGD is outside the scope of these works: the presented solutions do not account for Byzantine data nodes that can disrupt the training by sending erroneous gradients.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Differential Privacy and Byzantine Resilience in SGD

Guerraoui

Gupta

Pinot

et al. 2021

Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing

View full text Add to dashboard Cite

This paper addresses the problem of combining Byzantine resilience with privacy in machine learning (ML). Specifically, we study if a distributed implementation of the renowned Stochastic Gradient Descent (SGD) learning algorithm is feasible with both differential privacy (DP) and ( , )-Byzantine resilience. To the best of our knowledge, this is the first work to tackle this problem from a theoretical point of view. A key finding of our analyses is that the classical approaches to these two (seemingly) orthogonal issues are incompatible. More precisely, we show that a direct composition of these techniques makes the guarantees of the resulting SGD algorithm depend unfavourably upon the number of parameters of the ML model, making the training of large models practically infeasible. We validate our theoretical results through numerical experiments on publicly-available datasets; showing that it is impractical to ensure DP and Byzantine resilience simultaneously. CCS Concepts• Security and privacy → Privacy-preserving protocols; • Mathematics of computing → Continuous optimization.

show abstract

“…It is safe to assume that 2 + 2 + 2 ≥ 1. Thus, substituting from ( 16) in (18), we obtain that, for each ∈ {1, . .…”

Section: A Impracticality Results and Proofsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Differential Privacy and Byzantine Resilience in SGD

Guerraoui

Gupta

Pinot

et al. 2021

Proceedings of the 2021 ACM Symposium on Principles of Distributed Computing

View full text Add to dashboard Cite

show abstract

“…Other existing technical approaches also prevent a passive attacker from violating the privacy of data and information leakage by exploiting the global model outputs using malicious model updates. For example, the work in [236] presents a novel cryptographic key generation and sharing approach that leverages additive homomorphic encryption to maximize the confidentiality of federated gradient descent in the training of deep neural networks without any loss of accuracy.…”

Section: Security and Privacy Challengesmentioning

confidence: 99%

Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directions

Rauniyar¹,

Hagos²,

Jha³

et al. 2022

Preprint

View full text Add to dashboard Cite

With the advent of the Internet of Things (IoT), Artificial Intelligence (AI), and Machine Learning (ML)/Deep Learning (DL) algorithms, the data-driven medical application has emerged as a promising tool for designing reliable and scalable diagnostic and prognostic models from medical data. Hence, in recent years, data-driven medical applications have attracted a great deal of attention from academia to industry. This has undoubtedly improved the quality of healthcare delivery. However, AI-based medical applications still have poor adoption due to their difficulties in satisfying strict security, privacy, and quality of service standards such as low latency. Recent developments in Federated Learning (FL) have made it possible to train complex machine-learned models in a distributed manner and has become an active research domain, particularly processing the medical data at the edge of the network in a decentralized way to preserve privacy and address security concerns. To this end, this survey paper highlights the current and future of FL technology in medical applications where data sharing is a significant burden. It also review and discuss the current research trends and their outcomes for designing reliable and scalable FL models. We outline the general FL's statistical problems, device challenges, security, privacy concerns, and its potential in the medical domain. Moreover, our study is also focused on medical applications where we highlight the burden of global cancer and the efficient use of FL for the development of computer-aided diagnosis tools for addressing them. Additionally, the recent literature revealed that FL models ensure a high level of robustness and generalization compared to the traditional data-driven medical applications. We hope that this review serves as a checkpoint that sets forth the existing state-of-the-art works in a thorough manner and offers open problems and future research directions for this field.

show abstract

“…Privacy issue in optimization has gained increasing attention in recent years, in both non-distributed settings [1,25,53,94,96] and distributed settings [57,70,77,107,119]. In distributed settings, some research proposes encryption-based methods to prevent passive attackers from intercepting the exchanged information between agents in the network [70,107], while others utilizes differential privacy [77,119], a gold standard notion for privacy-preserving in data [31].…”

Section: Fault-tolerance and Privacymentioning

confidence: 99%

A Survey on Fault-tolerance in Distributed Optimization and Machine Learning

Liu

2021

Preprint

View full text Add to dashboard Cite

The robustness of distributed optimization is an emerging field of study, motivated by various applications of distributed optimization including distributed machine learning, distributed sensing, and swarm robotics. With the rapid expansion of the scale of distributed systems, resilient distributed algorithms for optimization are needed, in order to mitigate system failures, communication issues, or even malicious attacks. This survey investigates the current state of fault-tolerance research in distributed optimization, and aims to provide an overview of the existing studies on both fault-tolerant distributed optimization theories and applicable algorithms.

show abstract

MYSTIKO: Cloud-Mediated, Private, Federated Gradient Descent

Cited by 9 publications

References 16 publications

Differential Privacy and Byzantine Resilience in SGD

Differential Privacy and Byzantine Resilience in SGD

Federated Learning for Medical Applications: A Taxonomy, Current Trends, Challenges, and Future Research Directions

A Survey on Fault-tolerance in Distributed Optimization and Machine Learning

Contact Info

Product

Resources

About