Stable recovery of sparse vectors from random sinusoidal feature maps

Abstract

Random sinusoidal features are a popular approach for speeding up kernel-based inference in large datasets. Prior to the inference stage, the approach suggests performing dimensionality reduction by first multiplying each data vector by a random Gaussian matrix, and then computing an element-wise sinusoid. Theoretical analysis shows that collecting a sufficient number of such features can be reliably used for subsequent inference in kernel classification and regression.In this work, we demonstrate that with a mild increase in the dimension of the embedding, it is also possible to reconstruct the data vector from such random sinusoidal features, provided that the underlying data is sparse enough. In particular, we propose a numerically stable algorithm for reconstructing the data vector given the nonlinear features, and analyze its sample complexity. Our algorithm can be extended to other types of structured inverse problems, such as demixing a pair of sparse (but incoherent) vectors. We support the efficacy of our approach via numerical experiments.

Reconstruction from periodic nonlinearities, with applications to HDR imaging

“…As discussed in [5], the block diagonal structure of D and C allows the signal reconstruction problem to be reduced to a sequence of decoupled scalar estimation problems; such a decoupling enables the estimation of each entry of z and u independently of all other entries. We also assume that the analog values of the input signal lies within a range known a priori.…”

“…There are several different ways of doing this, including the multi-shot UHDR method of [3]. Here, we describe a novel approach, based on the MF-Sparse algorithm of [5]. We assume that the entries of z belong to some bounded set Ω ∈ R. Fix l ∈ [q] and form θ = exp(i u).…”

Demixing structured superposition signals from periodic and aperiodic nonlinear observations

“…In the periodic case, we use the approach of [17] both for designing the matrix X and the link function g. Specifically, we let X be factorized as X = DB, where D ∈ R m×q , and B ∈ R q×n have some specific structures; please see Section 2 for details. Again, for this case, we demonstrate a novel two-stage algorithm that stably estimate the components θ1 and θ2.…”

“…In this section, we focus on the periodic link functions which are either sinusoidal (complex-exponential), or any periodic function such that it is monotonic within each period. We start with the sinusoidal (complex-exponential) link function and follow the approach of [17]. In [17], the authors proposed an algorithm called MF-Sparse for recovering an underlying signal which is arbitrary sparse, or is the superposition of two arbitrary sparse components, but they only considered sinusoidal link functions.…”

“…By combining Theorem 4 and Theorem 2.1 in [17], we obtain the sample complexity of the MF-STRUCT-DHT scheme to achieve κ-accuracy. Note that big-Oh constant does not depend on the bounds on the derivative of the link function since it is a identity function.…”