Perfect Lp Sampling in a Data Stream

Abstract

In this paper, we resolve the one-pass space complexity of perfect L p sampling for p ∈ (0, 2) in a stream. Given a stream of updates (insertions and deletions) to the coordinates of an underlying vector f ∈ R n , a perfect L p sampler must output an index i with probability |f i | p / f p p , and is allowed to fail with some probability δ. So far, for p > 0 no algorithm has been shown to solve the problem exactly using poly(log n)-bits of space. In 2010, Monemizadeh and Woodruff introduced an approximate L p sampler, which outputs i with probability (1 ± ν)|f i | p / f p p , using space polynomial in ν −1 and log(n). The space complexity was later reduced by Jowhari, Saglam, and Tardos to roughly O(ν −p log 2 n log δ −1 ) for p ∈ (0, 2), which matches the Ω(log 2 n log δ −1 ) lower bound in terms of n and δ, but is loose in terms of ν.Given these nearly tight bounds, it is perhaps surprising that no lower bound exists in terms of ν-not even a bound of Ω(ν −1 ) is known. In this paper, we explain this phenomenon by demonstrating the existence of an O(log 2 n log δ −1 )-bit perfect L p sampler for p ∈ (0, 2). This shows that ν need not factor into the space of an L p sampler, which closes the complexity of the problem for this range of p. For p = 2, our bound is O(log 3 n log δ −1 )-bits, which matches the prior best known upper bound of O(ν −2 log 3 n log δ −1 ), but has no dependence on ν. For p < 2, our bound holds in the random oracle model, matching the lower bounds in that model. Moreover, we show that our algorithm can be derandomized with only a O((log log n) 2 ) blowup in the space (and no blow-up for p = 2). Our derandomization technique is quite general, and can be used to derandomize a large class of linear sketches, including the more accurate count-sketch variant of [MP14], resolving an open question in that paper.Finally, we show that a (1±ǫ) relative error estimate of the frequency f i of the sampled index i can be obtained using an additional O(ǫ −p log n)-bits of space for p < 2, and O(ǫ −2 log 2 n) bits for p = 2, which was possible before only by running the prior algorithms with ν = ǫ.

L p Samplers and Their Applications

“…Considering the results of [51], this resolves the space complexity of L p sampling for p ∈ (0, 2). For p = 2, the space complexity of [46] is O(log 3 n log(1/δ )) which matches the space of the best approximate sampler without dependence on ε.…”

“…Perfect L p Sampling. Recently, Jayaram and Woodruff [46] extended this approach to show space efficient perfect L p samplers. In other words, for p ∈ (0, 2) they give L p samplers with zero error in the sampling distribution (∆ = 0, ε = 0) using only O(log 2 n log(1/δ )) space.…”

New Streaming Algorithms for High Dimensional EMD and MST

“…When β ≥ 1 is not an integer, x −β is interpreted as x −⌊β⌋ . We will need the following lemma which bounds the tail ℓ 2 and ℓ 1 norm of a vector after its entries are scaled independently by random variables Lemma 5.6 (Generalization of Proposition 1 of [JW21]). Fix n ∈ AE and c ≥ 0, and let D 1 , .…”

“…Theorem 10 (Perfect ℓ 1 -sampling [JW21]). For m ∈ AE, let c > 1 be an arbitrarily large constant and t = O(log 2 m).…”

Differentially-Private Multi-Party Sketching for Large-Scale Statistics

“…A sequence of works showed that it is possible to compute a linear sketch of X that allows one to perform approximate L p sampling [17]. Such sketches can be used to perform moment estimation [45,54] and entropy estimation [13]. L 0 sampling is used for graph sketching, as discussed below.…”