High SNR Consistent Linear Model Order Selection and Subset Selection

IEEE Trans. Signal Process.

2019

Self Cite

Model order selection (MOS) in linear regression models is a widely studied problem in signal processing. Techniques based on information theoretic criteria (ITC) are algorithms of choice in MOS problems. This article proposes a novel technique called residual ratio thresholding for MOS in linear regression models which is fundamentally different from the ITC based MOS criteria widely discussed in literature. This article also provides a rigorous mathematical analysis of the high signal to noise ratio (SNR) and large sample size behaviour of RRT. RRT is numerically shown to deliver a highly competitive performance when compared to popular model order selection criteria like Akaike information criterion (AIC), Bayesian information criterion (BIC), penalised adaptive likelihood (PAL) etc. especially when the sample size is small.

Section: B High Snr Inconsistency Of Rrtmentioning

confidence: 94%

Section: B Prior Art On Mosmentioning

confidence: 99%

“…ITC based MOS estimate in (1) satisfies the following consistency results. [13], [14] a). Suppose that h(k, σ 2 k ) = vk and maximum model order p is fixed.…”

Section: B Prior Art On Mosmentioning

confidence: 99%

Section: B Prior Art On Mosmentioning

confidence: 99%

See 2 more Smart Citations

Residual Ratio Thresholding for Linear Model Order Selection

IEEE Trans. Signal Process.

2019

Self Cite

“…Then using Lemma 4 , r k Alg = (I n − P A k Alg )y = (I n − P A k Alg )g out + (I n − P A k Alg )w for k < k g and r k Alg = (I n − P A k Alg )w for k ≥ k g . Using standard distributional results discussed in [22] for deterministic projection matrices give the following for k > k g and σ 2 > 0.…”

Section: A Projection Matrices and Distributionsmentioning

confidence: 99%

Noise Statistics Oblivious GARD For Robust Regression With Sparse Outliers

IEEE Trans. Signal Process.

2019

Self Cite

Linear regression models contaminated by Gaussian noise (inlier) and possibly unbounded sparse outliers are common in many signal processing applications. Sparse recovery inspired robust regression (SRIRR) techniques are shown to deliver high quality estimation performance in such regression models. Unfortunately, most SRIRR techniques assume a priori knowledge of noise statistics like inlier noise variance or outlier statistics like number of outliers. Both inlier and outlier noise statistics are rarely known a priori and this limits the efficient operation of many SRIRR algorithms. This article proposes a novel noise statistics oblivious algorithm called residual ratio thresholding GARD (RRT-GARD) for robust regression in the presence of sparse outliers. RRT-GARD is developed by modifying the recently proposed noise statistics dependent greedy algorithm for robust de-noising (GARD). Both finite sample and asymptotic analytical results indicate that RRT-GARD performs nearly similar to GARD with a priori knowledge of noise statistics. Numerical simulations in real and synthetic data sets also point to the highly competitive performance of RRT-GARD. Index Terms: Robust regression, Sparse outliers, Greedy algorithm for robust regressionStep 1:-Identify the strongest residual in r k−1 GARD , i.e.,Step 2:-Update the matrixStep 3:-Estimate β and gout(S k GARD ) as [β Tĝ out(S k GARD ) T ] T = A k † y.Step 4:-Update the residual r k GARD = y − A k [β Tĝ out(S k GARD ) T ] T = (I n − P A k )y. k ← k + 1.Output:-Signal estimateβ. Outlier support estimate S k GARD .

High SNR consistent compressive sensing

2018

Signal Processing

Self Cite

Abstract-High signal to noise ratio (SNR) consistency of model selection criteria in linear regression models has attracted a lot of attention recently. However, most of the existing literature on high SNR consistency deals with model order selection. Further, the limited literature available on the high SNR consistency of subset selection procedures (SSPs) is applicable to linear regression with full rank measurement matrices only. Hence, the performance of SSPs used in underdetermined linear models (a.k.a compressive sensing (CS) algorithms) at high SNR is largely unknown. This paper fills this gap by deriving necessary and sufficient conditions for the high SNR consistency of popular CS algorithms like l0-minimization, basis pursuit denoising or LASSO, orthogonal matching pursuit and Dantzig selector. Necessary conditions analytically establish the high SNR inconsistency of CS algorithms when used with the tuning parameters discussed in literature. Novel tuning parameters with SNR adaptations are developed using the sufficient conditions and the choice of SNR adaptations are discussed analytically using convergence rate analysis. CS algorithms with the proposed tuning parameters are numerically shown to be high SNR consistent and outperform existing tuning parameters in the moderate to high SNR regime.