Learning Diagonal Gaussian Mixture Models and Incomplete Tensor Decompositions

Abstract

This paper studies how to learn parameters in diagonal Gaussian mixture models. The problem can be formulated as computing incomplete symmetric tensor decompositions. We use generating polynomials to compute incomplete symmetric tensor decompositions and approximations. Then the tensor approximation method is used to learn diagonal Gaussian mixture models. We also do the stability analysis. When the first and third order moments are sufficiently accurate, we show that the obtained parameters for the Gaussian mixture models are also highly accurate. Numerical experiments are also provided.

Algebraic compressed sensing

“…Only recently, the well-posedness of this problem was studied by Lindberg, Amendola, and Rodriguez [42]. For mixtures of Gaussians, Guo, Nie and Yang [43] recently presented an efficient numerical algorithm to estimate the parameters from coordinate projections of the third-order moment.…”

Distributionally Robust Optimization with Moment Ambiguity Sets

“…For the moment-based ambiguity, the set M is usually specified by the first, second moments [11,17,50]. Recently, higher order moments are also often used [8,15,28], especially in relevant applications with machine learning. For discrepancy-based ambiguity sets, popular examples are the φdivergence ambiguity sets [2,31] and the Wasserstein ambiguity sets [40].…”

Algebraic compressed sensing

“…Theorem 1.1 below shows that it suffices to understand the dimension of moment varieties to know the number of required measurements of the moment. For mixtures of Gaussians, Guo, Nie and Yang [47] recently presented an efficient numerical algorithm to estimate the parameters from coordinate projections of the third-order moment.…”