Robust One-Bit Recovery via ReLU Generative Networks: Near-Optimal Statistical Rate and Global Landscape Analysis

Abstract

We study the robust one-bit compressed sensing problem whose goal is to design an algorithm that faithfully recovers any sparse target vector θ 0 ∈ R d uniformly m quantized noisy measurements. Under the assumption that the measurements are sub-Gaussian random vectors, to recover any ksparse θ 0 (k ≪ d) uniformly up to an error ε with high probability, the best known computationally tractable algorithm requires 1 m ≥ O(k log d/ε 4 ) measurements. In this paper, we consider a new framework for the one-bit sensing problem where the sparsity is implicitly enforced via mapping a low dimensional representation x 0 ∈ R k through a known n-layer ReLU generative network G : R k → R d . Such a framework poses low-dimensional priors on θ 0 without a known basis. We propose to recover the target G(x 0 ) via an unconstrained empirical risk minimization (ERM) problem under a much weaker sub-exponential measurement assumption. For such a problem, we establish a joint statistical and computational analysis. In particular, we prove that the ERM estimator in this new framework achieves an improved statistical rate of m = O(kn log d/ε 2 ) recovering any G(x 0 ) uniformly up to an error ε. Moreover, from the lens of computation, despite non-convexity, we prove that the objective of our ERM problem has no spurious stationary point, that is, any stationary point are equally good for recovering the true target up to scaling with a certain accuracy. Furthermore, our analysis also shed lights on the possibility of inverting a deep generative model under partial and quantized measurements, complementing the recent success of using deep generative models for inverse problems.

Nonasymptotic Guarantees for Spiked Matrix Recovery with Generative Priors

“…Recently, in the wake of successes of deep learning , deep generative networks have gained popularity as a novel approach to encoding and enforcing priors. They have been successfully used as a prior for various statistical estimation problems such as compressed sensing [13], blind deconvolution [5], inpainting [61], and many more [54,60,51,59], etc.…”

“…Parallel to these empirical successes, a recent line of works have investigated theoretical guarantees for various statistical estimation tasks with generative network priors. Following the work of [13], [29] have given global guarantees for compressed sensing, followed then by many others for various inverse problems [53,44,25,6,51]. In particular [27] have shown that m = Ω(k log n) number of measurements are sufficient to recover a signal from random phaseless observations, assuming that the signal is the output of a trained generative network with latent space of dimension k. Note that, contrary to the sparse phase retrieval problem, generative priors for phase retrieval allow optimal sample complexity, up to logarithmic factors, with respect to the intrinsic dimension of the signal.…”

Constant-Expansion Suffices for Compressed Sensing with Generative Priors

“…In addition, there has been follow-up research on variations of the CS-DGP problem described above. The WDC is a critical assumption which enables efficient recovery in the setting of Gaussian noise [10], as well as global landscape analysis in the settings of phaseless measurements [8], one-bit (sign) measurements [17], two-layer convolutional neural networks [15], and low-rank matrix recovery [5]. Moreover, there are currently no known theoretical results in this area-compressed sensing with generative priors-that avoid the WDC: hence, until now, logarithmic expansion was necessary to achieve any provable guarantees.…”

“…One-bit recovery with a neural network prior, introduced in [17], has the following formal statement. Let G : R k → R n be a neural network of the form G(x) = ReLU(W (d) (.…”

“…Specifically, we observe a sign vector y = sign(AG(x * ) + ξ + τ) where A ∈ R m×n is a random measurement matrix, ξ ∈ R m is a noise vector, and τ ∈ R k is a random quantization threshold. In this setting, global landscape analysis of the loss function can be performed, and it can be shown that if each weight matrix satisfies the WDC then there are no spurious critical points outside a neighborhood of x * , a neighborhood of some negative scalar multiple of x * , and a neighborhood of 0 [17].…”