Improving inference of Gaussian mixtures using auxiliary variables

Abstract

Expanding a lower-dimensional problem to a higher-dimensional space and then projecting back is often beneficial. This article rigorously investigates this perspective in the context of finite mixture models, specifically how to improve inference for mixture models by using auxiliary variables. Despite the large literature in mixture models and several empirical examples, there is no previous work that gives general theoretical justification for including auxiliary variables in mixture models, even for special cases. We provide a theoretical basis for comparing inference for mixture multivariate models with the corresponding inference for marginal univariate mixture models. Analytical results for several special cases are established. We show that the probability of correctly allocating mixture memberships and the information number for the means of the primary outcome in a bivariate model with two Gaussian mixtures are generally larger than those in each univariate model. Simulations under a range of scenarios, including mis-specified models, are conducted to examine the improvement. The method is illustrated by two real applications in ecology and causal inference.

Exploiting multiple outcomes in Bayesian principal stratification analysis with application to the evaluation of a job training program

“…To simplify computation, we generate two continuous outcomes from a mixture of two bivariate normal distributions as model (4), and the stratum membership from a Bernoulli distribution as model (5). Although we only consider bivariate Normal distributions in our simulations, we can reasonably expect that our results are not tied to distributional assumptions: Mealli and Pacini (2013) show that secondary outcomes can also tighten large-sample nonparametric bounds for PCEs, and Mercatanti, Li and Mealli (2012) show that the use of an auxiliary variable may improve inference also in misspecified Gaussian mixture models. See also, for example, Gallop et al (2009), Mealli andPacini (2008), for further insights on the role of distributional assumptions in PS analysis.…”

“…As a second intuition, two (or more) distributions may be difficult to disentangle if they are similar, for example, if their means are very close; these same two means may instead be very far apart (and thus the mixture easier to disentangle) if considered in a two-dimensional space. In fact, recent theoretical results for mixture models [Mercatanti, Li and Mealli (2012)] show maining model parameters, while inferences for the finite sample causal estimands τ FS s would generally involve ρ regardless of the prior structure between parameters [for more discussion on this, see page 311 in Imbens and Rubin (1997)].…”

“…Multivariate analysis is beneficial for two reasons. First, models used in PS are inherently mixture models; recent results on mixture models show that with correct model specification, multivariate analysis leads to smaller variances of the parameters' estimators than those from a corresponding univariate analysis, resulting in more precise estimates [Mercatanti, Li and Mealli (2012)]. Second, some key substantive structural assumptions, such as exclusion restrictions, may be more plausible for secondary outcomes than for the primary one.…”

Using secondary outcome to sharpen bounds for treatment harm rate in characterizing heterogeneity

“…As mentioned in Mercatanti, Li, and Mealli (2015) , in some cases such as the mixture Gaussian situation, the introduction of a secondary outcome may induce extra uncertainty, so that it may lead to larger confidence interval than in the simpler modeling of only a primary outcome. But this will not happen in our case.…”

Joint and marginal causal effects for binary non-independent outcomes

“…Causal studies involving multivariate outcome variables are increasingly widespread in real-world applications: intervention studies in many fields routinely collect information on multiple outcomes. Recently, a strand of the causal inference literature has been working on using multiple outcomes, possible coupled with conditional independence assumptions, to address identification problems in causal studies with intermediate variables (Mattei et al, 2013;Mealli and Pacini, 2013;Mercatanti et al, 2015;Mealli et al, 2016). In these studies focus is on causal effects on a single response variable, which is viewed as the outcome of main interest, and additional outcomes are used as auxiliary variables for inferential purposes.…”