Sparse Approximation via Generating Point Sets

Abstract

For a set P of n points in the unit ball b ⊆ R d , consider the problem of finding a small subset T ⊆ P such that its convex-hull ε-approximates the convex-hull of the original set. Specifically, the Hausdorff distance between the convex hull of T and the convex hull of P should be at most ε. We present an efficient algorithm to compute such an ε -approximation of size k alg , where ε is a function of ε, and k alg is a function of the minimum size k opt of such an ε-approximation. Surprisingly, there is no dependence on the dimension d in either of the bounds. Furthermore, every point of P can be ε-approximated by a convex-combination of points of T that is O(1/ε 2 )-sparse.Our result can be viewed as a method for sparse, convex autoencoding: approximately representing the data in a compact way using sparse combinations of a small subset T of the original data. The new algorithm can be kernelized, and it preserves sparsity in the original input. * A preliminary version of this paper appeared in SODA 16 [BHR16]. The full version of the paper is also available on the arxiv [BHR15].

Robust vertex enumeration for convex hulls in high dimensions

“…Given thatS is γ robust and additionally γ is Ω(ε 1/3 ), then we cannot use fewer than |S| vertices to give an ε approximation. This argument shows that in Blum et al (2016) K opt = |S|. In a general case where γ is arbitrarily close to 0, AVTA will find all vertices in O(nK ε (m + 1 ε 2 )) time.…”

“…Thus approximation schemes have been studied for the problem. Blum et al (2016) propose a bi-criterion algorithm based on Nearest Neighbot Oracle, computing a subset of vertices T satisfying two properties: i) the Hausdorff distance between conv(T ) and conv(S) is bounded above by (8ε 1/3 + ε)R (ii) |T | = O(K opt /ε 2/3 ). Since T ⊂ S, this implies that ε = Ω((K opt /n) 3/2 ).…”

The discrete yet ubiquitous theorems of Carathéodory, Helly, Sperner, Tucker, and Tverberg

“…Theorem 3.5 can also be derived from Maurey's lemma in functional analysis (see Pisier [315] and Carl [90]). See also [72] for the derivation of a related theorem using the Perceptron algorithm [304]. A very recent new generalization of Theorem 3.5 was presented by Adiprasito, Bárány and Mustafa in [3].…”

Convex Hull Approximation of Nearly Optimal Lasso Solutions

“…We use the farthest point selection method, proposed in Blum et al [2016]. 4 In this procedure, we start from any point β 1 ∈ S. Then, we iteratively select the point β j ∈ S by solving the sample-approximated version of (4.1), i.e., the farthest point from the…”

“…Since we have no upper bound of the search range, we actually use the exponential search that successively doubles the search range[Bentley and Yao, 1976].4 Blum et al [2016] called this procedure Greedy Clustering.…”