We consider the problem of learning multi-ridge functions of the form f (x) = g(Ax) from point evaluations of f . We assume that the function f is defined on an 2 -ball in R d , g is twice continuously differentiable almost everywhere, and A ∈ R k×d is a rank k matrix, where k d. We propose a randomized, polynomial-complexity sampling scheme for estimating such functions. Our theoretical developments leverage recent techniques from low rank matrix recovery, which enables us to derive a polynomial time estimator of the function f along with uniform approximation guarantees. We prove that our scheme can also be applied for learning functions of the form:, provided f satisfies certain smoothness conditions in a neighborhood around the origin. We also characterize the noise robustness of the scheme. Finally, we present numerical examples to illustrate the theoretical bounds in action. recovery, randomized sampling, oracle-based learningKey words and phrases. Multi-ridge functions, high dimensional function approximation, low rank matrix recovery, non linear approximation, oracle-based learning.An extended abstract of this paper appeared in the 26 th Annual Conference on Neural Information Processing Systems (NIPS), December 2012. The present draft is an expanded version with a more rigorous analysis and consists of proofs of all the results.

No abstract

Abstract. Numerous dimensionality reduction problems in data analysis involve the recovery of lowdimensional models or the learning of manifolds underlying sets of data. Many manifold learning methods require the estimation of the tangent space of the manifold at a point from locally available data samples. Local sampling conditions such as (i) the size of the neighborhood (sampling width) and (ii) the number of samples in the neighborhood (sampling density) affect the performance of learning algorithms. In this work, we propose a theoretical analysis of local sampling conditions for the estimation of the tangent space at a point P lying on a m-dimensional Riemannian manifold S in R n . Assuming a smooth embedding of S in R n , we estimate the tangent space T P S by performing a Principal Component Analysis (PCA) on points sampled from the neighborhood of P on S. Our analysis explicitly takes into account the second order properties of the manifold at P , namely the principal curvatures as well as the higher order terms. We consider a random sampling framework and leverage recent results from random matrix theory to derive conditions on the sampling width and the local sampling density for an accurate estimation of tangent subspaces. We measure the estimation accuracy by the angle between the estimated tangent space T P S and the true tangent space T P S and we give conditions for this angle to be bounded with high probability. In particular, we observe that the local sampling conditions are highly dependent on the correlation between the components in the second-order local approximation of the manifold. We finally provide numerical simulations to validate our theoretical findings.

The convolution of a discrete measure, x " ř k i"1 aiδt i , with a local window function, φps´tq, is a common model for a measurement device whose resolution is substantially lower than that of the objects being observed. Super-resolution concerns localising the point sources tai, tiu k i"1 with an accuracy beyond the essential support of φps´tq, typically from m samples ypsjq " ř k i"1 aiφpsj´tiq`δj , where δj indicates an inexactness in the sample value. We consider the setting of x being non-negative and seek to characterise all non-negative measures approximately consistent with the samples. We first show that x is the unique non-negative measure consistent with the samples provided the samples are exact, i.e. δj " 0, m ě 2k`1 samples are available, and φps´tq generates a Chebyshev system. This is independent of how close the sample locations are and does not rely on any regulariser beyond non-negativity; as such, it extends and clarifies the work by Schiebinger et al. in [1] and De Castro et al. in [2], who achieve the same results but require a total variation regulariser, which we show is unnecessary.Moreover, we characterise non-negative solutionsx consistent with the samples within the bound ř m j"1 δ 2 j ď δ 2 . Any such non-negative measure is within Opδ 1{7 q of the discrete measure x generating the samples in the generalised Wasserstein distance. Similarly, we show using somewhat different techniques that the integrals ofx and x over pti´ǫ, ti`ǫq are similarly close, converging to one another as ǫ and δ approach zero. We also show how to make these general results, for windows that form a Chebyshev system, precise for the case of φps´tq being a Gaussian window. The main innovation of these results is that non-negativity alone is sufficient to localise point sources beyond the essential sensor resolution and that, while regularisers such as total variation might be particularly effective, they are not required in the non-negative setting.