Introduction to Coresets: Accurate Coresets

Abstract

A coreset (or core-set) of an input set is its small summation, such that solving a problem on the coreset as its input, provably yields the same result as solving the same problem on the original (full) set, for a given family of problems (models, classifiers, loss functions). Over the past decade, coreset construction algorithms have been suggested for many fundamental problems in e.g. machine/deep learning, computer vision, graphics, databases, and theoretical computer science. This introductory paper was written following requests from (usually non-expert, but also colleagues) regarding the many inconsistent coreset definitions, lack of available source code, the required deep theoretical background from different fields, and the dense papers that make it hard for beginners to apply coresets and develop new ones.The paper provides folklore, classic and simple results including step-by-step proofs and figures, for the simplest (accurate) coresets of very basic problems, such as: sum of vectors, minimum enclosing ball, SVD/ PCA and linear regression. Nevertheless, we did not find most of their constructions in the literature. Moreover, we expect that putting them together in a retrospective context would help the reader to grasp modern results that usually extend and generalize these fundamental observations. Experts might appreciate the unified notation and comparison table that links between existing results. Open source code with example scripts are provided for all the presented algorithms, to demonstrate their practical usage, and to support the readers who are more familiar with programming than math.

Introduction to Coresets: Approximated Mean

“…This survey is part of a series of surveys that aim to give introduction to coresets; see [Fel19] and [JMF19]. This work is organized as follows.…”

“…Those coresets do not introduce any error when compressing the data, i.e., ε = 0. The coresets presented in this section are explained in detail in [JMF19].…”

“…It turns out that we can compute in O(nd 2 ) time a coreset (P, u) of cardinality u 0 ≤ d + 2 where u ∈ R n that satisfies the conditions above. Furthermore, if the input weights w are nonnegative, i.e., w ∈ [0, ∞) n , we can compute in O(nd + d 4 log n) a coreset (P, u) of cardinality u 0 ≤ d + 3 where u ∈ [0, p∈P w(p)] d+3 is both non-negative and bounded; see full details in [JMF19].…”

“…Hence, we can compute a smaller subset of these d 2 -dimensional vectors, which accurately estimate their original sum. We thus obtain a weighted subset of the rows of A whose scatter matrix is the desired A T A, with no additional error [JMF19]. A coreset that introduces multiplicative 1 + ε error for this problem (SVD/linear regression) was suggested in [FVR16], also here the authors suggested a reduction to the problem of computing a mean coreset with multiplicative 1 + ε error for a set of point in a higher dimensional space.…”

“…Table 1: Summary of mean coresets. This table presents various coresets for the mean problem, both ε-coresets from this work and accurate coresets from [JMF19]. The input for all algorithms is a set P = {p1, • • • , pn} ⊆ R d and a weights vector w ∈ R n .…”