Maximum likelihood estimation for the tensor normal distribution: Algorithm, minimum sample size, and empirical bias and dispersion

Manceur, Ameur M.; Dutilleul, Pierre

doi:10.1016/j.cam.2012.09.017

Cited by 65 publications

(51 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, K are SPD matrices, usually normalized so that det P k = det Q k = 1 for each k and N = K k=1 n k [45]. One of the main advantages of the separable Kronecker model is the significant reduction in the number of variance-covariance parameters [42]. Usually, such separable covariance matrices are sparse and very large-scale.…”

Section: Multiway Divergences For Multivariate Normal Distributions Wmentioning

confidence: 99%

See 1 more Smart Citation

Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences

Cichocki

Cruces

Амари

2015

Entropy

View full text Add to dashboard Cite

This work reviews and extends a family of log-determinant (log-det) divergences for symmetric positive definite (SPD) matrices and discusses their fundamental properties. We show how to use parameterized Alpha-Beta (AB) and Gamma log-det divergences to generate many well-known divergences; in particular, we consider the Stein's loss, the S-divergence, also called Jensen-Bregman LogDet (JBLD) divergence, Logdet Zero (Bhattacharyya) divergence, Affine Invariant Riemannian Metric (AIRM), and other divergences. Moreover, we establish links and correspondences between log-det divergences and visualise them on an alpha-beta plane for various sets of parameters. We use this unifying framework to interpret and extend existing similarity measures for semidefinite covariance matrices in finite-dimensional Reproducing Kernel Hilbert Spaces (RKHS). This paper also shows how the Alpha-Beta family of log-det divergences relates to the divergences of multivariate and multilinear normal distributions. Closed form formulas are derived for Gamma divergences of two multivariate Gaussian densities; the special cases of the Kullback-Leibler, Bhattacharyya, Rényi, and Cauchy-Schwartz divergences are discussed. Symmetrized versions of log-det divergences are also considered and briefly reviewed. Finally, a class of divergences is extended to multiway divergences for separable covariance (or precision) matrices.

show abstract

Section: Multiway Divergences For Multivariate Normal Distributions Wmentioning

confidence: 99%

“…Recently, there has been growing interest in the analysis of tensors or multiway arrays [42][43][44][45]. One of the most important applications of multiway tensor analysis and multilinear distributions, is magnetic resonance imaging (MRI) (we refer to [46] and the references therein).…”

Section: Multiway Divergences For Multivariate Normal Distributions Wmentioning

confidence: 99%

Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences

Cichocki

Cruces

Амари

2015

Entropy

View full text Add to dashboard Cite

show abstract

“…Observe that the parameters Ψ and Σ are defined up to a positive multiplicative constant because, for example, I r ⊗ cΨ ⊗ c −1 Σ = I r ⊗ Ψ ⊗ Σ with c > 0. This issue has been discussed by some authors, among others (Dutilleul, 1999, Manceur and Dutilleul, 2013, Srivastava et al, 2008.…”

Section: Estimation In Trilinear Regression Modelmentioning

confidence: 93%

“…The Bayesian and the likelihood based approaches are the most used techniques to obtain estimators of unknown parameters in the tensor normal model, see for example (Hoff, 2011, Ohlson et al, 2013. For the third order tensor normal distribution the estimators can be found using the MLE-3D algorithm by Manceur and Dutilleul (2013) or similar algorithms like one proposed by Singull et al (2012). Let X i , i = 1, .…”

Section: It Follows Thatmentioning

confidence: 99%

Bilinear and Trilinear Regression Models with Structured Covariance Matrices

Nzabanita

2015

Linköping Studies in Science and Technology. Dissertations

View full text Add to dashboard Cite

Joseph Nzabanita (2015). Bilinear and Trilinear Regression Models with Structured Covariance Matrices Doctoral dissertation. This thesis focuses on the problem of estimating parameters in bilinear and trilinear regression models in which random errors are normally distributed. In these models the covariance matrix has a Kronecker product structure and some factor matrices may be linearly structured. Most of techniques in statistical modeling rely on the assumption that data were generated from the normal distribution. Whereas real data may not be exactly normal, the normal distributions serve as a useful approximation to the true distribution. The modeling of normally distributed data relies heavily on the estimation of the mean and the covariance matrix. The interest of considering various structures for the covariance matrices in different statistical models is partly driven by the idea that altering the covariance structure of a parametric model alters the variances of the model's estimated mean parameters.Firstly, we consider the extended growth curve model with a linearly structured covariance matrix. In general there is no problem to estimate the covariance matrix when it is completely unknown. However, problems arise when one has to take into account that there exists a structure generated by a few number of parameters. An estimation procedure that handles linear structured covariance matrices is proposed. The idea is first to estimate the covariance matrix when it may be used to define an inner product in a regression space and thereafter re-estimate it when it should be interpreted as a dispersion matrix. This idea is exploited by decomposing the residual space, the orthogonal complement to the design space, into orthogonal subspaces. Studying residuals obtained from projections of observations on these subspaces yields explicit consistent estimators of the covariance matrix. An explicit consistent estimator of the mean is also proposed.Secondly, we study a bilinear regression model with matrix normally distributed random errors. For those models, the dispersion matrix follows a Kronecker product structure and it can be used, for example, to model data with spatio-temporal relationships. The aim is to estimate the parameters of the model when, in addition, one covariance matrix is assumed to be linearly structured. On the basis of n independent observations from a matrix normal distribution, estimating equations, a flip-flop relation, are established.At last, the models based on normally distributed random third order tensors are studied. These models are useful in analyzing 3-dimensional data arrays. The 3-dimensional data arrays may be obtained when, for example, a single response is sampled in a 3-D space or in a 2-D space and time, multiple responses are recorded in a 2-D space or in a 1-D space and time. In some studies the analysis is done using the tensor normal model, where the focus is on the estimation of the variance-covariance matrix which has a Kronecker structure. Little attention...

show abstract

“…There is also a growing Bayesian and Frequentist literature on multiway array or tensor datasets, where this structure is commonly employed. See for example Akdemir and Gupta (2011), Allen (2012), Browne, MacCallum, Kim, Andersen, and Glaser (2002), Cohen, Usevich, and Comon (2016), Constantinou, Kokoszka, and Reimherr (2015), Dobra (2014), Fosdick and Hoff (2014), Gerard and Hoff (2015), Hoff (2011), Hoff (2015, Hoff (2016), Krijnen (2004), Leiva and Roy (2014), Leng and Tang (2012), Li and Zhang (2016), Manceura and Dutilleul (2013), Ning and Liu (2013), Ohlson, Ahmada, and von Rosen (2013), Singull, Ahmad, and von Rosen (2012), Volfovsky and Hoff (2014), Volfovsky and Hoff (2015), and Yin and Li (2012). In both these (apparently separate) literatures the dimension n is fixed and typically there are a small number of products each of whose dimension is of fixed but perhaps moderate size.…”

Section: Introductionmentioning

confidence: 99%

Estimation of a multiplicative covariance structure in the large dimensional case

Hafner

Linton

Tang

2016

View full text Add to dashboard Cite

We propose a Kronecker product structure for large covariance or correlation matrices. One feature of this model is that it scales logarithmically with dimension in the sense that the number of free parameters increases logarithmically with the dimension of the matrix. We propose an estimation method of the parameters based on a log-linear property of the structure, and also a quasi-maximum likelihood estimation (QMLE) method. We establish the rate of convergence of the estimated parameters when the size of the matrix diverges. We also establish a central limit theorem (CLT) for our method. We derive the asymptotic distributions of the estimators of the parameters of the spectral distribution of the Kronecker product correlation matrix, of the extreme logarithmic eigenvalues of this matrix, and of the variance of the minimum variance portfolio formed using this matrix. We also develop tools of inference including a test for over-identification. We apply our methods to portfolio choice for S&P500 daily returns and compare with sample covariance-based methods and with the recent Fan, Liao, and Mincheva (2013) method.

show abstract

Maximum likelihood estimation for the tensor normal distribution: Algorithm, minimum sample size, and empirical bias and dispersion

Cited by 65 publications

References 23 publications

Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences

Log-Determinant Divergences Revisited: Alpha-Beta and Gamma Log-Det Divergences

Bilinear and Trilinear Regression Models with Structured Covariance Matrices

Estimation of a multiplicative covariance structure in the large dimensional case

Contact Info

Product

Resources

About