Streaming k-means on Well-Clusterable Data

Abstract

One of the central problems in data-analysis is k-means clustering. In recent years, considerable attention in the literature addressed the streaming variant of this problem, culminating in a series of results (Har-Peled and Mazumdar; Frahling and Sohler; Frahling, Monemizadeh, and Sohler; Chen) that produced a (1 + ε)-approximation for k-means clustering in the streaming setting. Unfortunately, since optimizing the k-means objective is Max-SNP hard, all algorithms that achieve a (1 + ε)-approximation must take time exponential in k unless P=NP.Thus, to avoid exponential dependence on k, some additional assumptions must be made to guarantee high quality approximation and polynomial running time. A recent paper of Ostrovsky, Rabani, Schulman, and Swamy (FOCS 2006) introduced the very natural assumption of data separability: the assumption closely reflects how k-means is used in practice and allowed the authors to create a high-quality approximation for k-means clustering in the non-streaming setting with polynomial running time even for large values of k.Their work left open a natural and important question: are similar results possible in a streaming setting? This is the question we answer in this paper, albeit using substantially different techniques.We show a near-optimal streaming approximation algorithm for k-means in high-dimensional Euclidean space with sublinear memory and a single pass, under the same data separability assumption. Our algorithm offers significant improvements in both space and run- ning time over previous work while yielding asymptotically best-possible performance (assuming that the running time must be fully polynomial and P = NP).The novel techniques we develop along the way imply a number of additional results: we provide a high-probability performance guarantee for online facility location (in contrast, Meyerson's FOCS 2001 algorithm gave bounds only in expectation); we develop a constant approximation method for the general class of semi-metric clustering problems; we improve (even without σ-separability) by a logarithmic factor space requirements for streaming constant-approximation for k-median; finally we design a "re-sampling method" in a streaming setting to convert any constant approximation for clustering to a [1 + O(σ 2 )]-approximation for σ-separable data.

Deterministic sublinear-time approximations for metric 1-median selection

“…It is implicit in [23, Secs. 3.1-3.2] that metric k-median has a determinis- Many other algorithms are known for weighted or unweighted metric k-median selection [5,12,14,27,34], its online, streaming and non-metric variants [1,8,13,31,33] as well as the related problems of metric facility location [6,11,15,32,38], metric k-means selection [2,4,28] and allpairs shortest paths [7,9,[16][17][18]37]. All these algorithms, unlike ours, either randomize or inspect the entire available data.…”

Online Facility Location against a t-Bounded Adversary

“…Later, an algorithm storing only O(k log 2 n) points was provided [10]. The current state-of-the-art O(1)-approximation stores O(k log n) points [6]. A variety of other results are known for Euclidean space.…”

Proficient Centriod Selection Process for K-Mean Bunching Algorithm in Data Mining

“…The probability of K-assembles was showed up as before timetable as 1956 by Steinhaus [15]. A basic neighborhood heuristic for issue was proposed in 1957 by Lloyds [16].…”