A case for orthogonal measurements in linear inverse problems

Oymak, Samet; Hassibi, Babak

doi:10.1109/isit.2014.6875420

Cited by 15 publications

(25 citation statements)

References 21 publications

(37 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gaussian matrix from a Haar measure of random matrix. Our result further supports the argument of [26] that for the noisy setup (1) 4 and 5, one can realize that the divergence of the eigenvalues also results in performance degeneration.…”

Section: A Type-b Orthogonal Matrixsupporting

confidence: 90%

“…Gaussian matrix should be comparable. This inference also seems reasonable from the aspect of eigenvalue spectrum [26] that: when M N , an i.i.d. Gaussian matrix has approximately orthogonal rows and it behaves similar to a row-orthonormal matrix.…”

Section: B Type-c Matrixmentioning

confidence: 64%

“…2 This characteristics is in contrast to the noiseless setup mentioned above. Meanwhile, the authors of [26] supported the similar argument and further argued that one can still claim universality in the noisy setup if we restrict the measurement matrices to similar row-orthonormal type. These arguments showed that the choice of measurement matrices does have an impact in the MSE of reconstruction when noise is present.…”

Section: Introductionmentioning

confidence: 73%

See 2 more Smart Citations

On Sparse Vector Recovery Performance in Structurally Orthogonal Matrices via LASSO

Wen

Zhang

Wong

et al. 2016

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

In this paper, we consider a compressed sensing problem of reconstructing a sparse signal from an undersampled set of noisy linear measurements. The regularized least squares or least absolute shrinkage and selection operator (LASSO) formulation is used for signal estimation. The measurement matrix is assumed to be constructed by concatenating several randomly orthogonal bases, referred to as structurally orthogonal matrices. Such measurement matrix is highly relevant to large-scale compressive sensing applications because it facilitates fast computation and also supports parallel processing. Using the replica method from statistical physics, we derive the mean-squared-error (MSE) formula of reconstruction over the structurally orthogonal matrix in the large-system regime. Extensive numerical experiments are provided to verify the analytical result. We then use the analytical result to study the MSE behaviors of LASSO over the structurally orthogonal matrix, with a particular focus on performance comparisons to matrices with independent and identically distributed (i.i.d.) Gaussian entries. We demonstrate that the structurally orthogonal matrices are at least as well performed as their i.i.d. Gaussian counterparts, and therefore the use of structurally orthogonal matrices is highly motivated in practical applications. Index TermsCompressed sensing, LASSO, orthogonal measurement matrix, the replica method. ‡ C. Wen is with the Institute (negligible) vector addition. The complexity for evaluating the soft-thresholding function η is negligible.This kind of iterative thresholding algorithm requires few computations per-iteration, and therefore enables the application of LASSO in large-scale problems.Much of the theoretical work on (2) has focused on studying how aggressively a sparse signal can be undersampled while still guaranteeing perfect signal recovery. The existing results include those based on the restricted isometry property (RIP) [1,8], polyhedral geometry [9, 10], message passing [11], and the replica method [12][13][14]. Although RIP provides sufficient conditions for sparse signal reconstruction, the results provided by RIP analysis are often conservative in practice. In contrast, using combinational geometry, message passing, or the replica method, it is possible to compute the exact necessary and sufficient condition for measuring the sparsity-undersampling tradeoff performance of (2) in the limit N → ∞. However, the theoretical work largely focused on the case of having a measurement matrix A with independent and identically distributed (i.i.d.) entries. A natural question would be "how does the choice of the measurement matrix affect the typical sparsity-undersampling tradeoff performance?".There are strong reasons to consider different types of measurement matrix. Although the proximal gradient method performs efficiently in systems of medium size, the implementation of (4) will become prohibitively complex if the signal size is very large. This is not only because performing (4) requires matrix-ve...

show abstract

Section: A Type-b Orthogonal Matrixsupporting

confidence: 90%

Section: B Type-c Matrixmentioning

confidence: 64%

Section: Introductionmentioning

confidence: 73%

See 1 more Smart Citation

On Sparse Vector Recovery Performance in Structurally Orthogonal Matrices via LASSO

Wen

Zhang

Wong

et al. 2016

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…We deviate from the approach of [28], [30] by evaluating the performance directly using the replica method as in [31]- [34]. The derivations are 2 After the initial submission of the present paper, parallel studies using completely different mathematical methods and arguing for the superiority of the orthogonal constructions have been presented in [53] and [54]. Since then, an extension to the present paper has been proposed in [55] and iterative algorithms approximating Bayesian optimal estimation for structured matrices have been devised, see for example, [56]- [59].…”

Section: Contribution and Summary Of Resultsmentioning

confidence: 99%

Analysis of regularized LS reconstruction and random matrix ensembles in compressed sensing

Vehkaperä

Kabashima

Chatterjee

2014

2014 IEEE International Symposium on Information Theory

View full text Add to dashboard Cite

Abstract-Performance of regularized least-squares estimation in noisy compressed sensing is analyzed in the limit when the dimensions of the measurement matrix grow large. The sensing matrix is considered to be from a class of random ensembles that encloses as special cases standard Gaussian, roworthogonal, geometric and so-called T -orthogonal constructions. Source vectors that have non-uniform sparsity are included in the system model. Regularization based on 1-norm and leading to LASSO estimation, or basis pursuit denoising, is given the main emphasis in the analysis. Extensions to 2-norm and "zero-norm" regularization are also briefly discussed. The analysis is carried out using the replica method in conjunction with some novel matrix integration results. Numerical experiments for LASSO are provided to verify the accuracy of the analytical results.The numerical experiments show that for noisy compressed sensing, the standard Gaussian ensemble is a suboptimal choice for the measurement matrix. Orthogonal constructions provide a superior performance in all considered scenarios and are easier to implement in practical applications. It is also discovered that for non-uniform sparsity patterns the T -orthogonal matrices can further improve the mean square error behavior of the reconstruction when the noise level is not too high. However, as the additive noise becomes more prominent in the system, the simple row-orthogonal measurement matrix appears to be the best choice out of the considered ensembles.

show abstract

“…BP updates 2MN messages using (29), (30), (36), and (37) (i = 1, 2, · · · N, µ = 1, 2, · · · M) in each iteration. This requires a computational cost of O(M 2 ×N +M ×N 2 ) per iteration, which may limit the practical utility of BP to systems of relatively small size.…”

Section: Bayesian Optimal Signal Reconstruction By Gampmentioning

confidence: 99%

Bayesian signal reconstruction for 1-bit compressed sensing

Kabashima

Zdeborová

2014

J. Stat. Mech.

View full text Add to dashboard Cite

Abstract. The 1-bit compressed sensing framework enables the recovery of a sparse vector x from the sign information of each entry of its linear transformation. Discarding the amplitude information can significantly reduce the amount of data, which is highly beneficial in practical applications. In this paper, we present a Bayesian approach to signal reconstruction for 1-bit compressed sensing, and analyze its typical performance using statistical mechanics. As a basic setup, we consider the case that the measuring matrix Φ has i.i.d entries, and the measurements y are noiseless. Utilizing the replica method, we show that the Bayesian approach enables better reconstruction than the l 1 -norm minimization approach, asymptotically saturating the performance obtained when the non-zero entries positions of the signal are known, for signals whose nonzero entries follow zero mean Gaussian distributions. We also test a message passing algorithm for signal reconstruction on the basis of belief propagation. The results of numerical experiments are consistent with those of the theoretical analysis.

show abstract

A case for orthogonal measurements in linear inverse problems

Cited by 15 publications

References 21 publications

On Sparse Vector Recovery Performance in Structurally Orthogonal Matrices via LASSO

On Sparse Vector Recovery Performance in Structurally Orthogonal Matrices via LASSO

Analysis of regularized LS reconstruction and random matrix ensembles in compressed sensing

Bayesian signal reconstruction for 1-bit compressed sensing

Contact Info

Product

Resources

About