New Frameworks for Offline and Streaming Coreset Constructions

Abstract

Let P be a set (called points), Q be a set (called queries) and a function f : P ×Q → [0, ∞) (called cost). For an error parameter > 0, a set S ⊆ P with a weight function w : P → [0, ∞) is an ε-coreset if s∈S w(s)f (s, q) approximates p∈P f (p, q) up to a multiplicative factor of 1 ± ε for every given query q ∈ Q. Coresets are used to solve fundamental problems in machine learning of streaming and distributed data.We construct coresets for the k-means clustering of n input points, both in an arbitrary metric space and d-dimensional Euclidean space. For Euclidean space, we present the first coreset whose size is simultaneously independent of both d and n. In particular, this is the first coreset of size o(n) for a stream of n sparse points in a d ≥ n dimensional space (e.g. adjacency matrices of graphs). We also provide the first generalizations of such coresets for handling outliers. For arbitrary metric spaces, we improve the dependence on k to k log k and present a matching lower bound.For M -estimator clustering (special cases include the well-known k-median and k-means clustering), we introduce a new technique for converting an offline coreset construction to the streaming setting. Our method yields streaming coreset algorithms requiring the storage of O(S + k log n) points, where S is the size of the offline coreset. In comparison, the previous state-of-the-art was the merge-and-reduce technique that required O(S log 2a+1 n) points, where a is the exponent in the offline construction's dependence on −1 . For example, combining our offline and streaming results, we produce a streaming metric k-means coreset algorithm using O( −2 k log k log n) points of storage. The previous state-of-the-art required O( −4 k log k log 6 n) points.

Coresets for Decision Trees of Signals

“…If û assigns fractional weights, then some elements of B might be assigned to more than one label, as long as the sum of weights over every element in B is 1. By construction, ( Ŝ, ŵ) satisfies Properties ( 10)- (11). Hence, ( Ŝ, ŵ) is a smoothed version of Ĉ, û).…”

“…Furthermore, a coreset for a family of classifiers is many times a "silver bullet" that provides a unified solution to all Challenges (i)-(iv) above. Combining the two main coreset properties: merge and reduce [32,7,26,1], which are usually satisfied, with the fact that a coreset approximates every model, and not just the optimal model, enables it to support streaming and distributed data [11,41], parallel computation [21], handle constrained versions of the problem [25], model compression [20], parameter tuning [43] and more.…”

“…Recently, many works focus on developing frameworks for general families of loss functions, e.g., [22,57]. We refer the interested reader to the surveys [2,1,50,11,4,21] with references therein.…”

“…Let (S, w) be a pair which is a smoothed version of ( Ĉ, û). By definition of (S, w), we have that w sums to 1 over all (a, b) ∈ S with the same a, as in (11). Therefore, for every…”

Coresets for $(k, \ell)$-Median Clustering under the Fréchet Distance

“…In result, they address a spectrum of clustering problems, such as k-median clustering, k-line median clustering, projective clustering and also other problems like subspace approximation. Braverman et al [3] improved the aforementioned framework by switching to (ε, η)-approximations, which leads to substantially smaller sample sizes in many cases. Also, they simplified and further generalized the framework and applied it to k-means clustering of points in R d .…”