Enforcing Privacy in Distributed Learning with Performance Guarantees

Abstract

We study the privatization of distributed learning and optimization strategies. We focus on differential privacy schemes and study their effect on performance. We show that the popular additive random perturbation scheme degrades performance because it is not well-tuned to the graph structure. For this reason, we exploit two alternative graph-homomorphic constructions and show that they improve performance while guaranteeing privacy. Moreover, contrary to most earlier studies, the gradient of the risks is not assumed to be bounded (a condition that rarely holds in practice; e.g., quadratic risk). We avoid this condition and still devise a differentially private scheme with high probability. We examine optimization and learning scenarios and illustrate the theoretical findings through simulations.

Communication-Efficient and Privacy-Aware Distributed Learning

“…The privacy-accuracy tradeoff can be improved by perturbing the shared information using correlated noise sequences with decaying variances [19]- [21]. In [23], a graph topology-aware noise injection-based distributed learning strategy is introduced. It effectively cancels out the added noise during local aggregation steps, thereby enhancing the performance and privacy of the distributed learning algorithm.…”