Sequential reservoir sampling with a nonuniform distribution

Abstract

We present a simple algorithm that allows sampling from a stream of data items without knowing the number of items in advance and without having to store all items in main memory. The sampling distribution may be general, that is, the probability of selecting a data item i may depend on the individual item. The main advantage of the algorithms is that they have to pass through the data items only once to produce a sample of arbitrary size n .We give different variants of the algorithm for sampling with and without replacement and analyze their complexity. We generalize earlier results of Knuth on reservoir sampling with a uniform sampling distribution. The general distribution considered here allows us to sample an item with a probability equal to the relative weight (or fitness) of the data item within the whole set of items. Applications include heuristic optimization procedures such as genetic algorithms where solutions are sampled from a population with probability proportional to their fitness.

Non-sparse linear representations for visual tracking with online reservoir metric learning

“…Ideally, the sample buffers should keep a balance between sample diversity and adaptability. Motivated by this, reservoir sampling [18][19][20][21] is proposed for sequential random sampling. In principle, it aims to randomly draw some samples from a large population of samples that come in a sequential manner.…”

“…Therefore, larger weights should be assigned to the recently added samples while smaller weights should be attached with the old samples. Inspired by [20,21], we design a time-weighted reservoir sampling (TWRS) method for randomly drawing the samples according to their time-varying properties, as listed in Algorithm 3. The designed TWRS method is capable of effectively maintaining the sample buffers for online metric learning in Sec.…”

“…To allow for real-time applications, we design a timeweighted reservoir sampling method to maintain and update limited-sized sample buffers for balancing between sample diversity and adaptability in the metric learning procedure. With the theory of [20,21], larger weights are assigned to those recently received samples, which is particularly important for tracking. To our knowledge, it is the first time that reservoir sampling is used in an online metric learning setting that is tailored for robust visual tracking.…”

Weighted Random sampling based hierarchical amnesic synopses for data streams

“…Although we can directly sampling the weighted random sample R from D, but because the needs of sample merge which will be discussed later, we exploit a weighted random sampling method on data stream (called WRS, meaning Weighted Reservoir Sampling, refer to [12]) to obtain R.…”

Optimal sampling from sliding windows

“…Sampling is a very natural way to summarize data properties with sublinear space; indeed, it is a key component of many streaming algorithms and techniques. Just to mention a few, the relevant papers include Aggarwal [ [10]; BarYossef [13]; Bar-Yossef, Kumar and Sivakumar [17]; Buriol, Frahling, Leonardi, Marchetti-Spaccamela and Sohler [20]; Chakrabarti, Cormode and McGregor [21]; Chaudhuri and Mishra [26]; Chaudhuri, Motwani and Narasayya [27]; Cohen [29]; Cohen and Kaplan [30]; Cormode, Muthukrishnan and Rozenbaum [32]; Dasgupta, Drineas, Harb, Kumar and Mahoney [35]; Datar and Muthukrishnan [37]; Duffield, Lund and Thorup [38]; Frahling, Indyk and Sohler [43]; Gandhi, Suri and Welzl [46]; Gemulla [47]; Gemulla and Lehner [48]; Gibbons and Matias [49]; Guha, Meyerson, Mishra, Motwani and O'Callaghan [54]; Haas [55]; Kolonko and Wäsch [58]; Li [62]; Palmer and Faloutsos [67]; Szegedy [70]; and Vitter [72]; These papers illustrate the vitality of effective sampling methods for data streams. Among other methods, uniform random sampling is the most general and well-understood.…”