Structural learning of Bayesian networks using local algorithms based on the space of orderings

Abstract

Structural learning of Bayesian networks (BNs)is an NP-hard problem which is generally addressed by means of heuristic search algorithms. Despite the fact that earlier proposals for dealing with this task were based on searching the space of Directed Acyclic Graphs (DAGs), there are some alternative approaches. One of these approaches for structural learning consists of searching the space of orderings, as given a certain topological order among the problem variables, it is relatively easy to build (and evaluate) a BN compatible with it. In practice, the latter methods make it possible to obtain good results, but they are still costly in terms of computation. In this article, we prove the correctness of the method used to evaluate each ordering, and we propose some efficient learning algorithms based on it. Our first proposal is based on the Hill-Climbing algorithm, and uses an improved neighbourhood definition. The second algorithm is an extension of the first one, and is based on the well-known Variable Neighbourhood Search metaheuristic. Finally, iterative versions of both algorithms are also proposed. The algorithms have been tested over a set of different domains, and have been compared with other methods such as HillClimbing in the space of DAGs or Greedy Equivalent Search, in order to study their behaviour in practice.

A survey on Bayesian network structure learning from data

“…OBS employs a naïve operator, called swap, for escaping local maxima in the local greedy search. A more powerful operator, called insertion, was proposed by Alonso-Barba et al [4]. With this operator, the authors were able to significantly increase the score of the learned structures.…”

Optimizing regularized Cholesky score for order-based learning of Bayesian networks

“…The reason for naming (9) the Cholesky loss is that it provides an interesting variational characterization of the Cholesky factor of the inverse of a matrix as the following proposition shows. Let L p be the set of p×p lower triangular matrices with positive diagonal entries, and for any positive definite matrix M , let C(M ) be its unique Cholesky factor, i.e., the unique lower triangular matrix L with positive diagonal entries such that M = LL .…”

“…Substituting L * = C(A −1 ) into L chol (L; A), we obtain the minimum value: where L chol (L; A) is the Cholesky loss defined in (9).…”

“…junction with various operators that propose moves in the ordering space [8,9,10,11]. A greedy search, however, may easily be trapped in a local minimum, and thus different techniques were proposed to perform a more global search [7,12,13,14].…”

Clustering based on Bayesian networks with Gaussian and angular predictors : applications in Neuroscience

“…Successive changes are then applied to this order with the aim of optimising the network score. Given a set of operators over the orders, changes can be made locally using greedy search methods [Alonso-Barba et al, 2011;Cooper and Herskovits, 1992;Scanagatta et al, 2017;Teyssier and Koller, 2005] or some metaheuristics [Faulkner, 2007;Hsu et al, 2002;Larrañaga et al, 1996]. Their main disadvantages are that, without restrictions, there are as many orderings as permutations of variables, so the complexity in the worst case scenario is O(L!…”