Interaction is Necessary for Distributed Learning with Privacy or Communication Constraints

Dagan, Yuval; Feldman, Vitaly

doi:10.29012/jpc.781

Cited by 6 publications

(18 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Shamir [41] studies various estimation tasks under a range of information constraints. Finally, Dagan and Feldman [42] establish a separation between interactive and noninteractive learning for large-margin classifiers, under both local privacy and communication constraints.…”

Section: Prior Workmentioning

confidence: 99%

Interactive Inference under Information Constraints

Acharya¹,

Canonne²,

Liu³

et al. 2020

Preprint

View full text Add to dashboard Cite

We consider distributed inference using sequentially interactive protocols. We obtain lower bounds on the minimax sample complexity of interactive protocols under local information constraints, a broad family of resource constraints which captures communication constraints, local differential privacy, and noisy binary queries as special cases. We focus on the inference tasks of learning (density estimation) and identity testing (goodness-of-fit) for discrete distributions under total variation distance, and establish general lower bounds that take into account the local constraints modeled as a channel family.Our main technical contribution is an approach to handle the correlation that builds due to interactivity and quantifies how effectively one can exploit this correlation in spite of the local constraints. Using this, we fill gaps in some prior works and characterize the optimal sample complexity of learning and testing discrete distributions under total variation distance, for both communication and local differential privacy constraints. Prior to our work, this was known only for the problem of testing under local privacy constraints [1,2]. Our results show that interactivity does not help for learning or testing under these two constraints. Finally, we provide the first instance of a natural family of "leaky query" local constraints under which interactive protocols strictly outperform the noninteractive ones for distribution testing.

show abstract

Section: Prior Workmentioning

confidence: 99%

Interactive Inference under Information Constraints

Acharya¹,

Canonne²,

Liu³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The impact of private training on optimization Many recent works have studied the impact of differentially private mechanisms on centralized optimization [26,27,28,61,81,90,231]. Such works are often concerned with developing better differential privacy mechanisms for model training, and for getting tight bounds on the ( , δ) privacy guarantees of such mechanisms.…”

Section: Data Anonymizationmentioning

confidence: 99%

A Field Guide to Federated Optimization

Wang¹,

Charles²,

Xu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection. The distributed learning process can be formulated as solving federated optimization problems, which emphasize communication efficiency, data heterogeneity, compatibility with privacy and system requirements, and other constraints that are not primary considerations in other problem settings. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms through concrete examples and practical implementation, with a focus on conducting effective simulations to infer real-world performance. The goal of this work is not to survey the current literature, but to inspire researchers and practitioners to design federated learning algorithms that can be used in various practical applications.

show abstract

“…[40] studies various estimation tasks under a range of information constraints. Finally, [19] establish a separation between interactive and noninteractive learning for large-margin classifiers, under both local privacy and communication constraints.…”

Section: ) Interactive Testing and Estimation Of Discrete Distributionsmentioning

confidence: 99%

“…The crux of the previous bound is Eq. (19), which relates the Kullback-Leibler divergence to a per-coordinate information quantity…”

Section: Remark 18 (Is the Bound Above Tight?)mentioning

confidence: 99%

Interactive Inference Under Information Constraints

Acharya

Canonne

Liu

et al. 2022

IEEE Trans. Inform. Theory

View full text Add to dashboard Cite

We study the role of interactivity in distributed statistical inference under information constraints, e.g., communication constraints and local differential privacy. We focus on the tasks of goodness-of-fit testing and estimation of discrete distributions. From prior work, these tasks are well understood under noninteractive protocols. Extending these approaches directly for interactive protocols is difficult due to correlations that can build due to interactivity; in fact, gaps can be found in prior claims of tight bounds of distribution estimation using interactive protocols. We propose a new approach to handle this correlation and establish a unified method to establish lower bounds for both tasks. As an application, we obtain optimal bounds for both estimation and testing under local differential privacy and communication constraints. We also provide an example of a natural testing problem where interactivity helps.

show abstract

Interaction is Necessary for Distributed Learning with Privacy or Communication Constraints

Cited by 6 publications

References 30 publications

Interactive Inference under Information Constraints

Interactive Inference under Information Constraints

A Field Guide to Federated Optimization

Interactive Inference Under Information Constraints

Contact Info

Product

Resources

About