Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs

Abstract

Although many algorithms have been designed to construct Bayesian network structures using different approaches and principles, they all employ only two methods: those based on independence criteria, and those based on a scoring function and a search procedure (although some methods combine the two). Within the score+search paradigm, the dominant approach uses local search methods in the space of directed acyclic graphs (DAGs), where the usual choices for defining the elementary modifications (local changes) that can be applied are arc addition, arc deletion, and arc reversal. In this paper, we propose a new local search method that uses a different search space, and which takes account of the concept of equivalence between network structures: restricted acyclic partially directed graphs (RPDAGs). In this way, the number of different configurations of the search space is reduced, thus improving efficiency. Moreover, although the final result must necessarily be a local optimum given the nature of the search method, the topology of the new search space, which avoids making early decisions about the directions of the arcs, may help to find better local optima than those obtained by searching in the DAG space. Detailed results of the evaluation of the proposed search method on several test problems, including the well-known Alarm Monitoring System, are also presented.

Learning Bayesian Network Classifiers: Searching in a Space of Partially Directed Acyclic Graphs

“…With respect to the search methods, the algorithms in this category have commonly used local search methods (Cooper & Herskovits, 1992;Heckerman, Geiger, & Chickering, 1995), due to the exponentially large size of the search space. However, there is a growing interest in other heuristic search methods (for references, see Acid & de Campos, 2003).…”

“…This feature reduces the size of the search space, which has a smoother landscape, and avoids some early decisions on arc directions. The price that these algorithms must pay for this reduction, namely that the evaluation of candidate structures does not take advantage of the property of decomposability of many scoring functions (thus rendering these algorithms less efficient), has recently been overcome (Acid & de Campos, 2003;Chickering, 2002).…”

“…Although there is a kind of PDAG that can be used to canonically represent independent-equivalence classes of DAGs (called completed PDAGs by Chickering, 1996), the non-canonical scheme of representation that we shall use is that of restricted PDAGs (RPDAGs) ( Acid & de Campos, 2003), which are considerably simpler than completed PDAGs. Let us introduce some additional notation and then the concept of RPDAG.…”

On using Bayesian networks for complexity reduction in decision trees

“…Methods for automatic induction of BN model generally fall into two different classes: methods based on the examination of conditional independence constraints that hold over the empirical probability distributions on the variables represented in the data (also called Constraint-methods), and search methods that seek to maximize some scoring function that describes the ability of the network to explain the observed data (also called Score-methods). We concentrate in the paper on the latter approach, which aims to find the highest scoring BN model and may produce more accurate results in structure learning that Constraint-methods (Cooper and Herskovits 1993;Acid and de Campos 2003). The Score-methods are typically based on defining (1) a scoring function for evaluating the quality of a given structure, and (2) a search procedure for traversing the space of candidate models.…”

“…Several algorithms have been developed or proposed for identifying the Markov blanket (Margaritis and Thrun 1999;Frey et al 2003;) and the idea of using Markov blanket methods for feature selection is not new. For example, see references (Acid and de Campos 2003;Cowell et al 1999;Frey et al 2003) in Aliferis et al (2003). In particular, a Markov blanket based variable selection algorithm, named HITON, has been presented : it has been applied in combination with DTs, but also with other common classifiers, on several massive databases and it has been compared with some state-of-the-art variable selection methods in terms of classification accuracy.…”