Deterministic Coresets for k-Means of Big Sparse Data

Abstract

Let P be a set of n points in R d , k ≥ 1 be an integer and ε ∈ ( 0 , 1 ) be a constant. An ε-coreset is a subset C ⊆ P with appropriate non-negative weights (scalars), that approximates any given set Q ⊆ R d of k centers. That is, the sum of squared distances over every point in P to its closest point in Q is the same, up to a factor of 1 ± ε to the weighted sum of C to the same k centers. If the coreset is small, we can solve problems such as k-means clustering or its variants (e.g., discrete k-means, where the centers are restricted to be in P, or other restricted zones) on the small coreset to get faster provable approximations. Moreover, it is known that such coreset support streaming, dynamic and distributed data using the classic merge-reduce trees. The fact that the coreset is a subset implies that it preserves the sparsity of the data. However, existing such coresets are randomized and their size has at least linear dependency on the dimension d. We suggest the first such coreset of size independent of d. This is also the first deterministic coreset construction whose resulting size is not exponential in d. Extensive experimental results and benchmarks are provided on public datasets, including the first coreset of the English Wikipedia using Amazon’s cloud.

Survey on Technique and User Profiling in Unsupervised Machine Learning Method

“…[31] used a two-stage strategy to clustering, using hierarchical and non-hierarchical clustering, and came to the same conclusion about getting better outcomes. [32] suggested using self-organizing charts (i.e. model-oriented) to evaluate the clusters using the k-means algorithm.…”

Overview of accurate coresets

“…However, these coresets are exponential in the dimension of the input. Recently a deterministic coreset of size independent of was suggested (Barger & Feldman, 2020).…”

Introduction to Coresets: Approximated Mean

“…In particular, we can always improve a given k-clustering, by replacing the center of each cluster by its mean (if this is not already the case). This is indeed the idea behind the classic Lloyd's heuristic [Llo82] and also behind some coresets for k-means [BF20]. Most coreset construction algorithms for those hard problems usually borrow or generalize tricks and techniques used in coreset constructions for the (simpler) mean problem.…”

“…A coreset that introduces multiplicative 1 + ε error for this problem (SVD/linear regression) was suggested in [FVR16], also here the authors suggested a reduction to the problem of computing a mean coreset with multiplicative 1 + ε error for a set of point in a higher dimensional space. Another example is in the context of k-means, where [BF20] showed that in order to compute a k-means coreset for a set of points P it is suffices to cluster these points to a large number of clusters, and compute a mean coreset for each cluster, then take the union of these coresets to a single unite set, which is proven to be a k-means coreset for P .…”