On the theoretical analysis of cross validation in compressive sensing

Abstract

Compressive sensing (CS) is a data acquisition technique that measures sparse or compressible signals at a sampling rate lower than their Nyquist rate. Results show that sparse signals can be reconstructed using greedy algorithms, often requiring prior knowledge such as the signal sparsity or the noise level. As a substitute to prior knowledge, cross validation (CV), a statistical method that examines whether a model overfits its data, has been proposed to determine the stopping condition of greedy algorithms. This paper analyses cross validation in a general compressive sensing framework. Furthermore, we provide both theoretical analysis and numerical simulations for a cross-validation modification of orthogonal matching pursuit, referred to as OMP-CV, which has good performance in sparse recovery. ICASSP 2014This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. that sparse signals can be reconstructed using greedy algorithms, often requiring prior knowledge such as the signal sparsity or the noise level. As a substitute to prior knowledge, cross validation (CV), a statistical method that examines whether a model overfits its data, has been proposed to determine the stopping condition of greedy algorithms. This paper analyses cross validation in a general compressive sensing framework. Furthermore, we provide both theoretical analysis and numerical simulations for a cross-validation modification of orthogonal matching pursuit, referred to as OMP-CV, which has good performance in sparse recovery.

MMP-DCD-CV based sparse channel estimation algorithm for underwater acoustic transform domain communication system

“…To address aforementioned problems accounted in sparse reconstruction by using greedy algorithms such OMP etc., motivated by [50,52,54,55,56,57,58,59,60], we propose a new MMP algorithm that combines the DCD iterations and CV. Compared to the OMP algorithm, the MMP algorithm keeps and examines multiple promising candidate support sets rather than retaining only a single path set.…”

“…The uncertainty on sparsity and noise level can be bypassed using the CV. CV is a statistical method that can check whether the model is correct or not, and avoids underfitting and overfitting of data [54,55,56,58,59,60,57]. Cross validation can be used for stopping greedy iterations without the prior information such as sparsity or noise level.…”

“…According to the theory behind CV, the received vector y p in (10) is divided into the reconstruction vector y re p and CV vector y cv p . Correspondingly, the matrix C is also split into two sub-matrices, a reconstruction matrix C re ∈ C m×L and a CV matrix C cv ∈ C mcv×L with m cv = M − m. Then one has a reconstruction equation [55,56,57,58,59,60] y re p = C re h + n re p ,…”

“…In this paper, a novel MMP based sparse channel estimation algorithm with dichotomous coordinate descent (DCD) iterations [51] and CV [56,54,55,58,59,60,57], henceforth referred to as MMP-DCD-CV, is proposed for UWA TDCSs. Our contributions are as follows: 1) We propose a modification to MMP based on the line search with DCD iterations (MMP-DCD).…”

Multistatic Airborne Passive Synthetic Aperture Radar Imaging Based on Two-Level Block Sparsity

“…If the change of the recover error is no longer significant with the increase of the value of K, the current value of K can be used as the input parameter of the proposed two-level BMP algorithm. The cross validation technique [23,24] can also be an alternative way to estimate the appropriate sparsity level of observed scene. Table 1.…”