Tight Sensitivity Bounds For Smaller Coresets

Maalouf, Alaa; Statman, Adiel; Feldman, Dan

doi:10.48550/arxiv.1907.01433

Cited by 2 publications

(5 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then Lemma 8 completes the proof of the theorem for subspaces. The reduction from affine subspaces to proper subspaces can be found in Section 4 of [12].…”

Section: Methodsmentioning

confidence: 99%

“…2. A non-uniform sampling algorithm, Algorithm 1 of [12], that guarantees a tight bound of sensitivity for any k and exact sensitivity for d = k − 1. Sampling points according to their sensitivities guarantees a coreset with an error of in a size of O( d log 2 d…”

Section: P (|Mmentioning

confidence: 99%

“…For example, given a large clustering problem over n points, one could build an -coreset of O(log n) points and then run an off-the-shelf algorithm on the coreset to obtain a (1 + )-approximate solution. Coresets have been widely studied in recent years for a variety of problems [7,1,12,9].…”

Section: Introductionmentioning

confidence: 99%

“…We consider Singular Value Decomposition (SVD) in the streaming model, which is applicable to processing large datasets or real-time data [10,12]. In the streaming model, the rows of a (possibly infinite) matrix A ∈ R ?×d arrive sequentially as (a 1 , a 2 , a 3 , .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Sparse Coresets for SVD on Infinite Streams

Braverman¹,

Feldman²,

Lang³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

In streaming Singular Value Decomposition (SVD), d-dimensional rows of a possibly infinite matrix arrive sequentially as points in R d . An -coreset is a (much smaller) matrix whose sum of square distances of the rows to any hyperplane approximates that of the original matrix to a 1 ± factor. Our main result is that we can maintain a -coreset while storing only O(d log 2 d/ 2 ) rows. Known lower bounds of Ω(d/ 2 ) rows show that this is nearly optimal. Moreover, each row of our coreset is a weighted subset of the input rows. This is highly desirable since it: (1) preserves sparsity; (2) is easily interpretable; (3) avoids precision errors; (4) applies to problems with constraints on the input. Previous streaming results for SVD that return a subset of the input required storing Ω(d log 3 n/ 2 ) rows where n is the number of rows seen so far. Our algorithm, with storage independent of n, is the first result that uses finite memory on infinite streams. We support our findings with experiments on the Wikipedia dataset benchmarked against state-of-the-art algorithms.

show abstract

“…Then Lemma 8 completes the proof of the theorem for subspaces. The reduction from affine subspaces to proper subspaces can be found in Section 4 of [12].…”

Section: Methodsmentioning

confidence: 99%

Section: P (|Mmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Sparse Coresets for SVD on Infinite Streams

Braverman¹,

Feldman²,

Lang³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Example coresets in machine learning include SVM [30,54,55,56,57], z -regression [15,18,51], clustering [2,13,21,28,34,39,50], logistic regression [32,44], LMS solvers and SVD [25,41,42,49], where all of these works have been dedicated to suggest a coreset for a specific problem.…”

Section: Modern Machine Learningmentioning

confidence: 99%

Coresets for Near-Convex Functions

Tukan,

Maalouf,

Feldman

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Coreset is usually a small weighted subset of n input points in R d , that provably approximates their loss function for a given set of queries (models, classifiers, etc.). Coresets become increasingly common in machine learning since existing heuristics or inefficient algorithms may be improved by running them possibly many times on the small coreset that can be maintained for streaming distributed data. Coresets can be obtained by sensitivity (importance) sampling, where its size is proportional to the total sum of sensitivities. Unfortunately, computing the sensitivity of each point is problem dependent and may be harder to compute than the original optimization problem at hand. We suggest a generic framework for computing sensitivities (and thus coresets) for wide family of loss functions which we call near-convex functions. This is by suggesting the f -SVD factorization that generalizes the SVD factorization of matrices to functions. Example applications include coresets that are either new or significantly improves previous results, such as SVM, Logistic regression, M-estimators, and z -regression. Experimental results and open source are also provided.Preprint. Under review.

show abstract

Tight Sensitivity Bounds For Smaller Coresets

Cited by 2 publications

References 31 publications

Sparse Coresets for SVD on Infinite Streams

Sparse Coresets for SVD on Infinite Streams

Coresets for Near-Convex Functions

Contact Info

Product

Resources

About