A MCMC Approach for Learning the Structure of Gaussian Acyclic Directed Mixed Graphs

Abstract

Graphical models are widely used to encode conditional independence constraints and causal assumptions, the directed acyclic graph (DAG) being one of the most common families of models. However, DAGs are not closed under marginalization: that is, if a distribution is Markov with respect to a DAG, several of its marginals might not be representable with another DAG unless one discards some of the structural independencies. Acyclic directed mixed graphs (ADMGs) generalize DAGs so that closure under marginalization is possible. In a previous work, we showed how to perform Bayesian inference to infer the posterior distribution of the parameters of a given Gaussian ADMG model, where the graph is fixed. In this paper, we extend this procedure to allow for priors over graph structures.Key words: Graphical models, Bayesian inference, sparsity. Acyclic Directed Mixed Graph ModelsDirected acyclic graphs (DAGs) provide a practical language to encode conditional independence constraints (see, e.g., [8]). However, such a family is not closed under marginalization. As an illustration of this concept, consider the following DAG:This model entails several conditional independencies. For instance, it encodes constraints such as Y 2 ⊥ ⊥ Y 4 , as well asDirected graphical models are non-monotonic independence models, in the sense that conditioning on extra variables can destroy and re-create independencies, as the sequence { / 0, {Y 3 }, {Y 3 , X}} has demonstrated.

Bayesian inference via projections

“…Figures 3 and 4 provide some illustration of the behavior of Bayesian projections. In Figures 3, our implied prior over model structures provides more diffuse posteriors than the one adopted by the more traditional Bayesian approach introduced by [20]. Both posteriors still centered close to the right model for data sets of size 10000, but Bayesian projections does still have a somewhat broad posterior, which in some sense reflects the greater insensitivity of the Frobenius norm compared to the likelihood function of the Gibbs procedure.…”

“…In [20], we introduce a Gibbs sampler for the graphical structure G in the Gaussian case. [23] introduces a new variation of the idea, where the graphical structure does not encode hard constraints: instead each edge represents a mixture indicator, with the lack of an edge representing a prior for σ ij strongly concentrated around zero, and the presence of an edge as indicating a high variance prior.…”

“…Although this prior puts zero probability on σ ij = 0, it allows the mixture indicators to be sampled independently within a Gibbs sampling step, increasing its computational efficiency. In our experiments, we used a modified version of our Gibbs sampler [20], which allows for positive mass on sparsity patterns 2 .…”

Gaussian covariance graph models accounting for correlated marker effects in genome‐wide prediction

“…In order to overcome this problem, Wang () proposed a method to perform covariance model selection with improved computational efficiency. On the other hand, Silva and Kalaitzis () developed an approach to improve the efficiency of MCMC algorithms used to perform Bayesian inference and showed its application in covariance model selection, and Silva () proposed a method based on acyclic directed mixed graphs (a generalization of directed acyclic graphs) that can be used to estimate the covariance matrix when the pattern of zeros is unknown. Some of these methods could be implemented in genome‐wide prediction following approaches similar to those presented in this study.…”

Bayesian inference for latent chain graphs

“…There are several contributions that propose Bayesian methods in related contexts. In [28], the author proposes a method to perform full Bayesian inference for acyclic directed mixed graphs using Markov chain Monte Carlo (MCMC) methods. There are several similarities between the model there (based upon structural equation models (SEM) e.g .…”