Fast Algorithms for Demixing Sparse Signals From Nonlinear Observations

Soltani, Mohammadreza; Hegde, Chinmay

doi:10.1109/tsp.2017.2706181

Cited by 19 publications

(31 citation statements)

References 59 publications

(139 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assume that link function g(x) is a differentiable monotonic function, satisfying 0 < µ 1 ≤ g (x) ≤ µ 2 for all x ∈ D(g) (domain of g). This assumption is standard in statistical learning [19] and in nonlinear sparse recovery [52,59,60]. Also, as we will discuss below, this assumption will be helpful for verifying the RSC/RSS condition for the loss function that we define as follows.…”

Section: Nonlinear Affine Rank Minimizationmentioning

confidence: 97%

Fast Low-Rank Matrix Estimation for Ill-Conditioned Matrices

Soltani

Hegde

2018

2018 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

In this paper, we study the general problem of optimizing a convex function F (L) over the set of p × p matrices, subject to rank constraints on L. However, existing first-order methods for solving such problems either are too slow to converge, or require multiple invocations of singular value decompositions. On the other hand, factorization-based non-convex algorithms, while being much faster, require stringent assumptions on the condition number of the optimum. In this paper, we provide a novel algorithmic framework that achieves the best of both worlds: asymptotically as fast as factorization methods, while requiring no dependency on the condition number.We instantiate our general framework for three important matrix estimation problems that impact several practical applications; (i) a nonlinear variant of affine rank minimization, (ii) logistic PCA, and (iii) precision matrix estimation in probabilistic graphical model learning. We then derive explicit bounds on the sample complexity as well as the running time of our approach, and show that it achieves the best possible bounds for both cases. We also provide an extensive range of experimental results, and demonstrate that our algorithm provides a very attractive tradeoff between estimation accuracy and running time.

show abstract

Section: Nonlinear Affine Rank Minimizationmentioning

confidence: 97%

Fast Low-Rank Matrix Estimation for Ill-Conditioned Matrices

Soltani

Hegde

2018

2018 IEEE International Symposium on Information Theory (ISIT)

Self Cite

View full text Add to dashboard Cite

show abstract

“…To overcome the inherent ambiguity issue in problem (1), many existing methods have assumed that the structures of sets (i.e., the structures can be low-rank matrices, or have sparse representation in some domain [McCoy and Tropp, 2014]) X and N are a prior known and also that the signals from X and N are "distinguishable" [Elad and Aharon, 2006, Soltani and Hegde, 2016, Soltani and Hegde, 2017, Druce et al, 2016, Elyaderani et al, 2017. The assumption of having the prior knowledge is a big restriction in many real-world applications.…”

Section: Application and Prior Artmentioning

confidence: 99%

Leaming Structured Signals Using GAN s with Applications in Denoising and Demixing

Soltani

Jain

Sambasivan

et al. 2019

2019 53rd Asilomar Conference on Signals, Systems, and Computers

Self Cite

View full text Add to dashboard Cite

Recently, Generative Adversarial Networks (GANs) have emerged as a popular alternative for modeling complex high dimensional distributions. Most of the existing works implicitly assume that the clean samples from the target distribution are easily available. However, in many applications, this assumption is violated. In this paper, we consider the observation setting when the samples from target distribution are given by the superposition of two structured components and leverage GANs for learning the structure of the components. We propose two novel frameworks: denoising-GAN and demixing-GAN. The denoising-GAN assumes access to clean samples from the second component and try to learn the other distribution, whereas demixing-GAN learns the distribution of the components at the same time. Through extensive numerical experiments, we demonstrate that proposed frameworks can generate clean samples from unknown distributions, and provide competitive performance in tasks such as denoising, demixing, and compressive sensing. 1 arXiv:1902.04664v1 [stat.ML] 12 Feb 2019 1.1 Setup and Our Technique Motivated by the recent success of generative models in high dimensional statistical inference tasks such as compressed sensing in [Bora et al., 2017, Bora et al., 2018, in this paper, we focus on Generative Adversarial Network (GAN) based generative models to implicitly learn the distributions, i.e., generate samples from their distributions. Most of the existing works on GANs typically assume access to clean samples from the underlying signal distribution. However, this assumption clearly breaks down in the superposition model considered in our setup, where the structured superposition makes training generative models very challenging.In this context, we investigate the first question with varying degrees of assumption about the access to clean samples from the two signal sources. We first focus on the setting when we have access to samples only from the constituent signal class N and observations, Y i 's. In this regard, we propose the denoising-GAN framework. However, assuming access to samples from one of the constituent signal class can be restrictive and is often not feasible in real-world applications. Hence, we further relax this assumption and consider the more challenging demixing problem, where samples from the second constituent component are not available and solve it using what we call the demixing-GAN framework.Finally, to answer the second question, we use our trained generator(s) from the proposed GAN frameworks for denoising and demixing tasks on unseen test samples (i.e., samples not used in the training process) by discovering the best hidden representation of the constituent components from the generative models. In addition to the denoising and demixing problems, we also consider a compressive sensing setting to test the trained generator(s). Below we explicitly list the contribution made in this paper:

show abstract

“…The goal is now to recover both G(z) and ν. This is reminiscent of the problem of source separation or signal demixing [20], and in our previous work [17], [21] we proposed greedy iterative algorithms for solving such demixing problems. We extend this work by proving a nonlinear extension, together with a new analysis, of the algorithm proposed in [17].…”

Section: Techniquesmentioning

confidence: 99%

Algorithmic Aspects of Inverse Problems Using Generative Models

Hegde

2018

2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton)

Self Cite

View full text Add to dashboard Cite

The traditional approach of hand-crafting priors (such as sparsity) for solving inverse problems is slowly being replaced by the use of richer learned priors (such as those modeled by generative adversarial networks, or GANs). In this work, we study the algorithmic aspects of such a learningbased approach from a theoretical perspective. For certain generative network architectures, we establish a simple nonconvex algorithmic approach that (a) theoretically enjoys linear convergence guarantees for certain inverse problems, and (b) empirically improves upon conventional techniques such as back-propagation. We also propose an extension of our approach that can handle model mismatch (i.e., situations where the generative network prior is not exactly applicable.) Together, our contributions serve as building blocks towards a more complete algorithmic understanding of generative models in inverse problems.

show abstract

Fast Algorithms for Demixing Sparse Signals From Nonlinear Observations

Cited by 19 publications

References 59 publications

Fast Low-Rank Matrix Estimation for Ill-Conditioned Matrices

Fast Low-Rank Matrix Estimation for Ill-Conditioned Matrices

Leaming Structured Signals Using GAN s with Applications in Denoising and Demixing

Algorithmic Aspects of Inverse Problems Using Generative Models

Contact Info

Product

Resources

About