On Estimating Frequency Moments of Data Streams

Abstract

Abstract. Space-economical estimation of the pth frequency moments, defined as Fp = P n i=1 |fi| p , for p > 0, are of interest in estimating all-pairs distances in a large data matrix [14], machine learning, and in data stream computation. Random sketches formed by the inner product of the frequency vector f1, . . . , fn with a suitably chosen random vector were pioneered by Alon, Matias and Szegedy [1], and have since played a central role in estimating Fp and for data stream computations in general. The concept of p-stable sketches formed by the inner product of the frequency vector with a random vector whose components are drawn from a p-stable distribution, was proposed by Indyk [11] for estimating Fp, for 0 < p < 2, and has been further studied in Li [13]. In this paper, we consider the problem of estimating Fp, for 0 < p < 2. A disadvantage of the stable sketches technique and its variants is that they require O( 1 2 ) inner-products of the frequency vector with dense vectors of stable (or nearly stable [14,13]) random variables to be maintained. This means that each stream update can be quite time-consuming. We present algorithms for estimating Fp, for 0 < p < 2, that does not require the use of stable sketches or its approximations. Our technique is elementary in nature, in that, it uses simple randomization in conjunction with well-known summary structures for data streams, such as the COUNT-MIN sketch [7] and the COUNTSKETCH structure [5]. Our algorithms require spaceÕ( 1 2+p ) 3 to estimate Fp to within 1 ± factors and requires expected time O(log F1 log 1 δ ) to process each update. Thus, our technique trades an O( 1 p ) factor in space for much more efficient processing of stream updates. We also present a stand-alone iterative estimator for F1.

Measuring independence of datasets

“…In their setting vector entries are defined by frequencies of the corresponding elements in the stream. Their influential paper was followed by a sequence of results including, among many others, works by Bhuvanagiri, Ganguly, Kesh and Saha [7]; Charikar, Chen and Farach-Colton [14]; Cormode and Muthukrishnan [15,16]; Feigenbaum, Kannan, Strauss and Viswanathan [17]; Ganguly and Cormode [18]; Indyk [20]; Indyk and Woodruff [22]; and Li [24] as well as work of authors [9,10,13].…”

On the Exact Space Complexity of Sketching and Streaming Small Norms

“…Other work on L p estimation for 0 < p ≤ 2 includes the work of Ganguly and Cormode [18], which requires a suboptimal O(ε −(2+p) log O(1) (mM )) bits of space, but at the benefit of requiring log O(1) (mM ) update time independent of ε.…”

Sketching and Streaming Entropy via Approximation Theory

“…Many algorithmic problems in this model are now well-understood, for example, the problem of estimating frequency moments [1,2,10,18,32,35]. More recently, several researchers have studied the problem of estimating the empirical entropy of a stream [3,6,7,12,13,37].…”