Low‐complexity cross‐validation design of a linear estimator

Tong, Jun; Xi, Jiangtao; Guo, Qinghua; Yu, Yanguang

doi:10.1049/el.2017.2364

Cited by 4 publications

(4 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, [51] derives analytical LOOCV solutions to minimize the MSE of the CM estimation for Gaussian data, [69] proposes a CV solution to optimize the inverse covariance matrix (a.k.a. precision matrix) estimation, while [21], [70] derive LOOCV solutions for the application of minimum mean square error (MMSE) filtering. In this paper, we extend these works to obtain data-driven and low-complexity approaches for choosing ρ for the MVDR beamformer, by finding a suitable CV cost as an estimate of J (Σ) in ( 6) and its fast evaluation.…”

Section: B Covariance (Scatter) Matrix Estimatormentioning

confidence: 99%

Cross-Validated Tuning of Shrinkage Factors for MVDR Beamforming Based on Regularized Covariance Matrix Estimation

Xie¹,

He²,

Tong³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper considers the regularized estimation of covariance matrices (CM) of high-dimensional (compound) Gaussian data for minimum variance distortionless response (MVDR) beamforming. Linear shrinkage is applied to improve the accuracy and condition number of the CM estimate for lowsample-support cases. We focus on data-driven techniques that automatically choose the linear shrinkage factors for shrinkage sample covariance matrix (S 2 CM) and shrinkage Tyler's estimator (STE) by exploiting cross validation (CV). We propose leaveone-out cross-validation (LOOCV) choices for the shrinkage factors to optimize the beamforming performance, referred to as S 2 CM-CV and STE-CV. The (weighted) out-of-sample output power of the beamfomer is chosen as a proxy of the beamformer performance and concise expressions of the LOOCV cost function are derived to allow fast optimization. For the large system regime, asymptotic approximations of the LOOCV cost functions are derived, yielding the S 2 CM-AE and STE-AE. In general, the proposed algorithms are able to achieve near-oracle performance in choosing the linear shrinkage factors for MVDR beamforming. Simulation results are provided for validating the proposed methods.

show abstract

Section: B Covariance (Scatter) Matrix Estimatormentioning

confidence: 99%

Cross-Validated Tuning of Shrinkage Factors for MVDR Beamforming Based on Regularized Covariance Matrix Estimation

Xie¹,

He²,

Tong³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…The grid search algorithm will find the best number of k in the range of 1 to 50 with the step of 1. To obtain the value of accuracy itself, a validation mechanism called K-fold Cross-Validation (CV) [15] with 5 fold, 10 fold, 20 fold, and leave one out (LOO) are used. Accuracy of a model can be calculated with the formula that is shown in equation 9 [16].…”

Section: Performance Measurement and Comparisonmentioning

confidence: 99%

A New Feature Extraction Algorithm to Extract Differentiate Information and Improve KNN-based Model Accuracy on Aquaculture Dataset

Natan

Gunawan

Dewantara

2019

Int. J. Adv. Sci. Eng. Inf. Technol.

View full text Add to dashboard Cite

In the world of aquaculture, understanding the condition of a pond is very important for a farmer in deciding which action should they take to prevent any bad condition occurred. Condition of a pond can be justified by measuring plenty of water parameters which can be divided into 3 categories that are physical, chemical and biological. The physical parameter is any physical quantity that can be measured in the pond. The chemical parameter is any kind of chemical substances that are dissolved in water. The biological parameter is any organic matter that lives in water. However, all of these parameters are not so distinguishable in representing the condition of a pond. Therefore, the farmer experience difficulties in justifying the condition and taking proper action to their pond. Even with the help of the K-Nearest Neighbors (KNN) algorithm combined with grid search optimization to model the data, the result is still not satisfying where the model only achieve accuracy of 0.701 in leave one out validation. To overcome this problem, a kind of feature extraction algorithm is needed to extract more information and make the data become more differentiate in representing the condition of the pond. With the help of our proposed feature extraction algorithm, optimized KNN can model the data easier and achieve higher accuracy. From the experiment results, the proposed feature extraction algorithm gives an impressive performance where it increases the accuracy to 0.741. A comparison with other feature extraction algorithms such as Principal Component Analysis (PCA), Non-negative Matrix Factorization (NMF), and Singular Value Decomposition (SVD) is also conducted to validate how good the proposed feature extraction algorithm is. As a result, the proposed algorithm is surpassing the other algorithms which only achieve the accuracy of 0.707, 0.718, and 0.718, respectively.

show abstract

“…Eqn. (47) can be exploited to compute the closed-form LOOCV solution quickly. From (47), the most involved computation for finding the solution of the optimization problem (15) can be implemented as…”

Section: Loocv Choice For Ols-based Covariance Estimationmentioning

confidence: 99%

“…Note that in contrast to the cross-validation methods in [35] and [36] which choose shrinkage factors by a grid search for optimizing the signal estimation performance, the method proposed in this paper has an analytical solution and optimizes covariance matrix estimation. It also differs from [47] which targets the design of a signal estimator that shrinks the sample LMMSE filter toward the matched filter. Example 6: Application to MVDR beamforming: Finally, we show an example application to minimum variance distortionless response (MVDR) beamforming [31], [33].…”

Section: Numerical Examplesmentioning

confidence: 99%

Linear shrinkage estimation of covariance matrices using low-complexity cross-validation

Tong

et al. 2018

Signal Processing

View full text Add to dashboard Cite

Shrinkage can effectively improve the condition number and accuracy of covariance matrix estimation, especially for low-sample-support applications with the number of training samples smaller than the dimensionality. This paper investigates parameter choice for linear shrinkage estimators. We propose data-driven, leave-one-out cross-validation (LOOCV) methods for automatically choosing the shrinkage coefficients, aiming to minimize the Frobenius norm of the estimation error. A quadratic loss is used as the prediction error for LOOCV. The resulting solutions can be found analytically or by solving optimization problems of small sizes and thus have low complexities. Our proposed methods are compared with various existing techniques. We show that the LOOCV method achieves near-oracle performance for shrinkage designs using sample covariance matrix (SCM) and several typical shrinkage targets. Furthermore, the LOOCV method provides low-complexity solutions for estimators that use general shrinkage targets, multiple targets, and/or ordinary least squares (OLS)-based covariance matrix estimation. We also show applications of our proposed techniques to several different problems in array signal processing.

show abstract

Low‐complexity cross‐validation design of a linear estimator

Cited by 4 publications

References 8 publications

Cross-Validated Tuning of Shrinkage Factors for MVDR Beamforming Based on Regularized Covariance Matrix Estimation

Cross-Validated Tuning of Shrinkage Factors for MVDR Beamforming Based on Regularized Covariance Matrix Estimation

A New Feature Extraction Algorithm to Extract Differentiate Information and Improve KNN-based Model Accuracy on Aquaculture Dataset

Linear shrinkage estimation of covariance matrices using low-complexity cross-validation

Contact Info

Product

Resources

About