Eigenvalues and constraints in mixture modeling: geometric and computational issues

Abstract

This paper presents a review about the usage of eigenvalues restrictions for constrained parameter estimation in mixtures of elliptical distributions according to the likelihood approach. These restrictions serve a twofold purpose: to avoid convergence to degenerate solutions and to reduce the onset of non interesting (spurious) maximizers, related to complex likelihood surfaces. The paper shows how the constraints may play a key role in the theory of Euclidean data clustering. The aim here is to provide a reasoned review of the constraints and their applications, along the contributions of many authors, spanning the literature of the last thirty years.Keywords Mixture Model · EM algorithm · Eigenvalues · Model-based clustering IntroductionFinite mixture distributions play a central role in statistical modelling, as they combine much of the flexibility of non parametric models with nice analytical properties of parametric models, see e.g.

Anomaly and Novelty detection for robust semi-supervised learning

“…From the seminal paper of Day (1969) we know that the likelihood for Gaussian mixture is unbounded and the ML approach is thus an ill posed problem, whenever constraints like in (4) are not assumed. Notice further that the constraint in (4) is still needed whenever either the shape or the volume is free to vary across components (García-Escudero et al, 2018): feasible algorithms for enforcing the eigen-ratio constraint under different specification of the covariance matrices as per Figure 2 are reported in Cappozzo et al (2019).…”

“…Given the semisupervised nature of the problem at hand, we propose to further rely on the information that can be extracted from the robustly initialized estimates { τ , μ, Σ} to set sensible values for the fixed constant c ≥ 1 required in the eigenvalue-ratio restriction. That is, if no prior information for the value c is available, as it is almost always the case in real applications (García-Escudero et al, 2018), the following quantity could be, al least initially, used:…”

“…They often display a high likelihood value, however, little insight can be obtained in real-world applications as they are mostly a result of modeling a localized random pattern rather than a true underlying group. Many possible solutions have been proposed in the literature to tackle the problem, a comprehensive list of such references can be found in García-Escudero et al (2018).…”

Constrained parsimonious model-based clustering

“…Starting from a seminal paper by Hathaway (1985), an alternative approach is to constrain the Σ j scatter matrices by specifying some tuning constants that control the strength of the constraints. A fairly comprehensive review of this approach can be found in García-Escudero et al (2018). For instance, a recent proposal following this idea is the "deter-and-shape" one in García-Escudero et al (2020).…”

Model-based clustering with determinant-and-shape constraint

“…In fact, non-interesting local maxima denoted as "spurious" solutions, which consist of a few, almost collinear, observations, are often detected by the Classification EM algorithm (CEM), traditionally applied when maximizing (1), and by the EM algorithm, traditionally applied when maximizing (2). A recent review of approaches for dealing with this lack of boundedness and for reducing the detection of spurious solutions can be found in García-Escudero et al (2018).…”