Estimating linear covariance models with numerical nonlinear algebra

Abstract

Numerical nonlinear algebra is applied to maximum likelihood estimation for Gaussian models defined by linear constraints on the covariance matrix. We examine the generic case as well as special models (e.g. Toeplitz, sparse, trees) that are of interest in statistics. We study the maximum likelihood degree and its dual analogue, and we introduce a new software package LinearCovarianceModels.jl for solving the score equations. All local maxima can thus be computed reliably. In addition we identify several scenarios for which the estimator is a rational function.

Maximum likelihood degree of the two-dimensional linear Gaussian covariance model

“…We have proven that for m = 2 and arbitrary n, the MLdegree is 2n − 3, which agrees with the computations in Table 1 of [15]. The authors of [15] further conjecture that for 3-dimensional models, the ML-degree is 3n 2 − 9n + 7 and that for 4-dimensional models, the ML-degree is 11/3n 3 − 18n 2 + 85/3n − 15. We believe that the methods used here will be useful for proving these conjectures as well as for approaching generic linear Gaussian covariance models of arbitrary dimension.…”

“…In [15], Sturmfels, Timme and Zwiernik use numerical algebraic geometry methods implemented in the Julia package LinearGaussianCovariance.jl to compute the MLdegrees of linear Gaussian covariance models for several values of n and m, where m is the dimension of model. We have proven that for m = 2 and arbitrary n, the MLdegree is 2n − 3, which agrees with the computations in Table 1 of [15]. The authors of [15] further conjecture that for 3-dimensional models, the ML-degree is 3n 2 − 9n + 7 and that for 4-dimensional models, the ML-degree is 11/3n 3 − 18n 2 + 85/3n − 15.…”

“…We believe that the methods used here will be useful for proving these conjectures as well as for approaching generic linear Gaussian covariance models of arbitrary dimension. The authors of [15] also compute dual ML-degrees of several linear Gaussian covariance…”

“…for generic A, B and S. Thus, by rearranging (1), using the Julia software package LinearCovarianceModels.jl [15]. We then generated a matrix with random entries from the interval [−1, 1], and multiplied the matrix by its transpose to obtain the sample covariance matrix: We used the numerical polynomial solver package HomotopyContinuation.jl [5] to find the solutions to the system of equations f = 0, g = 0.…”

“…This is the number of complex critical points of the log-likelihood function, and it is considered to be a measurement of the difficulty of computing the maximum likelihood estimate [15, Table 3]. Knowledge of the ML-degree of a model is important when applying numerical algebraic geometry methods to solve the MLE problem; in particular, it gives a stopping criterion for monodromy methods [15,Section 5]. For more background on ML-degrees, we refer the reader to [8, Chapter 2].…”