Covariance structure regularization via Frobenius-norm discrepancy

Abstract

In many practical problems, the underlying structure of an estimated covariance matrix is usually blurred due to random noise, particularly when the dimension of the matrix is high. Hence, it is necessary to filter the random noise or regularize the available covariance matrix in certain senses, so that the covariance structure becomes clear. In this paper, we propose a new method for regularizing the covariance structure of a given covariance matrix. By choosing an optimal structure from an available class of covariance structures, the regularization is made in terms of minimizing the discrepancy, defined by Frobenius-norm, between the given covariance matrix and the class of covariance structures. A range of potential candidate structures, including the order-1 moving average structure, compound symmetry structure, order-1 autoregressive structure, order-1 autoregressive moving average structure, are considered. Simulation studies show that the proposed new approach is reliable in regularization of covariance structures. The proposed approach is also applied to real data analysis in signal processing, showing the usefulness of the proposed approach in practice.

Covariance regularization for metabolomic data on the drought resistance of barley

“…Cui et al, 2016, p. 128 andLin et al, 2014, p. 317). The solution of this problem is considered in Cui et al (2016) for the Frobenius norm and in Lin et al (2014) for the entropy loss function. In the next part we introduce notation and present solutions for the case of each considered structure with the given discrepancy functions.…”

“…James and Stein, 1961;Dey and Srinivasan, 1985;Lin et al, 2014;Filipiak et al, 2018b;Filipiak et al, 2018c) as the discrepancy functions. Cui et al (2016) considered the square of the Frobenius norm, whilst in this paper the plain Frobenius norm is considered. Observe that since the Frobenius norm is a convex function, its square is also convex, and therefore the minimum is the same.…”

“…The matrix Ψ AR is positive definite for σ 2 ą 0 and belonging to the interval p´1; 1q, cf. Cui et al (2016). The AR(1) structure is a special case of Toeplitz matrices with p " m´1.…”

“…Such covariance structure selection is called regularization; cf. Cui et al (2016) and Lin et al (2014). Regularization consists of two steps.…”

Correlation structure regularization via entropy loss function for high-dimension and low-sample-size data

“…We therefore need to select an optimal value of λ via certain selection criteria. In the spirit of Cui et al (2016), we propose to minimize the Frobenius-norm discrepancy when selecting the optimal λ. Hence our regularized covariance estimator takes the advantages of both entropy loss and Frobenius-norm discrepancy in this sense.…”