Interactive Inference under Information Constraints

Abstract

We consider distributed inference using sequentially interactive protocols. We obtain lower bounds on the minimax sample complexity of interactive protocols under local information constraints, a broad family of resource constraints which captures communication constraints, local differential privacy, and noisy binary queries as special cases. We focus on the inference tasks of learning (density estimation) and identity testing (goodness-of-fit) for discrete distributions under total variation distance, and establish general lower bounds that take into account the local constraints modeled as a channel family.Our main technical contribution is an approach to handle the correlation that builds due to interactivity and quantifies how effectively one can exploit this correlation in spite of the local constraints. Using this, we fill gaps in some prior works and characterize the optimal sample complexity of learning and testing discrete distributions under total variation distance, for both communication and local differential privacy constraints. Prior to our work, this was known only for the problem of testing under local privacy constraints [1,2]. Our results show that interactivity does not help for learning or testing under these two constraints. Finally, we provide the first instance of a natural family of "leaky query" local constraints under which interactive protocols strictly outperform the noninteractive ones for distribution testing.

AHEAD: Adaptive Hierarchical Decomposition for Range Query under Local Differential Privacy

“…1) AHEAD is robust to the changes in data distributions. Recalling the derivation • 6 groups, while HDG separates users into 𝐶 2 3 +𝐶 1 3 groups, where the number of user records in each group of AHEAD is half of that of HDG, i.e., doubling noise error of AHEAD. Therefore, from Figure 8(a) and 8(b), we know that the superiority of AHEAD will decrease with fewer user records.…”

“…When answering a range query, HIO completely covers the query range by using the minimum number of intervals from different layers. For example, when the users' private attribute domain size |𝐷 | = 8 and tree fanout 𝐵 = 2, the range query [2,7] can be decomposed into intervals [2,3] ∪ [4,7]. Then, HIO adds the estimated frequency values of the two intervals above to get the answer of a range query.…”

“…When the assumption is not satisfied, it leads to a non-uniform error. For instance, if the questioner wants to know 𝑓 𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 ( [2,3]), the answer of the adaptive strategy is calculated from the frequency value of interval [0, 3], i.e., the half of the frequency value of interval [0, 3]. Compared with the baseline strategy, the adaptive strategy reduces the noise error from 𝜎 2 to 𝜎 2 2 , while it also brings nonuniform error 0.05 − 0.05 2 = 0.025.…”

“…Privacy Guarantee. AHEAD is sequentially interactive [2,15,35,36,38], i.e., each user communicates once (Step 2 in subsection 4.2) but the randomization depends on earlier user's messages (Step 3 in subsection 4.2). Since the private data of each user is transmitted to the aggregator once via OUE with privacy budget 𝜖 (no other information of the users is leaked), we claim that AHEAD rigorously satisfies 𝜖-LDP and the proof is deferred to Appendix B due to the space limitation.…”

“…𝑋 𝑖 represents the perturbed noise added to the frequency of the 𝑖-th sub-interval. For each layer, the number of users in Equation 2 is the same for all intervals, i.e., E[𝑋 2 𝑖 ] = 𝑂 (𝑐/𝑁 ) for any 𝑖. Leveraging the adaptive strategy used in AHEAD, we estimate the frequency value of node 𝑛 and assign the average as the child nodes' values. Then, we obtain the expected estimation error.…”