Streaming Algorithms via Precision Sampling

Andoni, Alexandr; Krauthgamer, Robert; Onak, Krzysztof

doi:10.1109/focs.2011.82

Cited by 90 publications

(120 citation statements)

References 36 publications

Supporting

Mentioning

119

Contrasting

Unclassified

Order By: Relevance

“…The estimation of frequency moments over a data stream has been the subject of much study over the past two decades, starting with the work of Alon, Matias and Szegedy [1]. See, e.g., the references in [3]. Our algorithms are the first small-space (in fact, the first sub-linear in m space) algorithms for estimating correlated frequency moments with provable guarantees on the relative error.…”

Section: Contributionsmentioning

confidence: 99%

See 1 more Smart Citation

A General Method for Estimating Correlated Aggregates Over a Data Stream

Tirthapura

Woodruff

2014

Algorithmica

View full text Add to dashboard Cite

On a stream S" role="presentation" style="box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">SS of two dimensional data items (x,y)" role="presentation" style="boxsizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">(x,y)(x,y) where x" role="presentation" style="box-sizing: border-box; display: inline-table; line-height: normal; letterspacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">xx is an item identifier and y" role="presentation" style="box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; minheight: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">yy is a numerical attribute, a correlated aggregate query C(σ,AGG,S)" role="presentation" style="box-sizing: border-box; display: inlinetable; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">C(σ,AGG,S)C(σ,AGG,S) asks to first apply a selection predicate σ" role="presentation" style="box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">σσ along the y" role="presentation" style="box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; minheight: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">yy dimension, followed by an aggregation AGG" role="presentation" style="box-sizing: border-box; display: inline-table; line-height: normal; letter-spacing: normal; word-spacing: normal; word-wrap: normal; white-space: nowrap; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border: 0px; padding: 0px; margin: 0px; position: relative;">AGGAGG along the x" role="presentation" style="...

show abstract

Section: Contributionsmentioning

confidence: 99%

“…2 Fix the random string of A for the rest of this algorithm. 3 In the first pass (ε, δ )-approximate f ymax using A , obtaining estimatef ymax . 4 Set r = log 1+εfymax .…”

Section: Algorithmmentioning

confidence: 99%

A General Method for Estimating Correlated Aggregates Over a Data Stream

Tirthapura

Woodruff

2014

Algorithmica

View full text Add to dashboard Cite

show abstract

“…When each input rectangle is a single point, this definition reduces to the classical definition of the kth frequency moment of a stream, which has been very well studied in the literature starting from the work of Alon, Matias, and Szegedy [2] 1 (see for example the references in the recent papers [3,9]). As shown by Braverman and Ostrovsky [10], techniques for understanding the frequency moments were used to characterize the class of all sketchable functions.…”

Section: Problem Definitionmentioning

confidence: 99%

“…One problem with this overall approach is that while it was recently discovered that g and f can be implemented with pairwise-independent hash functions [3,9] for estimating F k , it was not known that s can also be implemented with a pairwise-independent hash function. For example, the seminal work of Alon, Matias, and Szegedy [2] requires 4-wise independence for estimating F2 to within a constant factor.…”

Section: Our Techniquesmentioning

confidence: 99%

Rectangle-efficient aggregation in spatial data streams

Tirthapura

Woodruff

2012

Proceedings of the 31st ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems

View full text Add to dashboard Cite

We consider the estimation of aggregates over a data stream of multidimensional axis-aligned rectangles. Rectangles are a basic primitive object in spatial databases, and efficient aggregation of rectangles is a fundamental task. The data stream model has emerged as a de facto model for processing massive databases in which the data resides in external memory or the cloud and is streamed through main memory. For a point p, let n(p) denote the sum of the weights of all rectangles in the stream that contain p. We give nearoptimal solutions for basic problems, including (1) the k-th frequency moment F k = points p |n(p)| k , (2) the counting version of stabbing queries, which seeks an estimate of n(p) given p, and (3) identification of heavy-hitters, i.e., points p for which n(p) is large. An important special case of F k is F0, which corresponds to the volume of the union of the rectangles. This is a celebrated problem in computational geometry known as "Klee's measure problem", and our work yields the first solution in the streaming model for dimensions greater than one.

show abstract

“…It is not possible to prove a lower bound better than c a c v =Ω(n 1−2/k ) since there exist standard (Merlin-less) streaming algorithms for computing F k that useÕ(n 1−2/k ) space [7,21].…”

Section: Open Problemsmentioning

confidence: 99%

Practical verified computation with streaming interactive proofs

Cormode

Mitzenmacher

Thaler

2012

Proceedings of the 3rd Innovations in Theoretical Computer Science Conference

154

269

View full text Add to dashboard Cite

When delegating computation to a service provider, as in the cloud computing paradigm, we seek some reassurance that the output is correct and complete. Yet recomputing the output as a check is inefficient and expensive, and it may not even be feasible to store all the data locally. We are therefore interested in what can be validated by a streaming (sublinear space) user, who cannot store the full input, or perform the full computation herself. Our aim in this work is to advance a recent line of work on "proof systems" in which the service provider proves the correctness of its output to a user. The goal is to minimize the time and space costs of both parties in generating and checking the proof. Only very recently have there been attempts to implement such proof systems, and thus far these have been quite limited in functionality.Here, our approach is two-fold. First, we describe a carefully chosen instantiation of one of the most efficient general-purpose constructions for arbitrary computations (streaming or otherwise), due to Goldwasser, Kalai, and Rothblum [19]. This requires several new insights and enhancements to move the methodology from a theoretical result to a practical possibility. Our main contribution is in achieving a prover that runs in time O(S(n) log S(n)), where S(n) is the size of an arithmetic circuit computing the function of interest; this compares favorably to the poly(S(n)) runtime for the prover promised in [19]. Our experimental results demonstrate that a practical general-purpose protocol for verifiable computation may be significantly closer to reality than previously realized.Second, we describe a set of techniques that achieve genuine scalability for protocols fine-tuned for specific important problems in streaming and database processing. Focusing in particular on noninteractive protocols for problems ranging from matrix-vector multiplication to bipartite perfect matching, we build on prior work [8,15] to achieve a prover that runs in nearly linear-time, while obtaining optimal tradeoffs between communication cost and the user's working memory. Existing techniques required (substantially) superlinear time for the prover. Finally, we develop improved interactive protocols for specific problems based on a linearization technique originally due to Shen [34]. We argue that even if general-purpose methods improve, fine-tuned protocols will remain valuable in real-world settings for key problems, and hence special attention to specific problems is warranted.

show abstract

Streaming Algorithms via Precision Sampling

Cited by 90 publications

References 36 publications

A General Method for Estimating Correlated Aggregates Over a Data Stream

A General Method for Estimating Correlated Aggregates Over a Data Stream

Rectangle-efficient aggregation in spatial data streams

Practical verified computation with streaming interactive proofs

Contact Info

Product

Resources

About