Time and sample efficient discovery of Markov blankets and direct causal relations

Abstract

Data Mining with Bayesian Network learning has two important characteristics: under broad conditions learned edges between variables correspond to causal influences, and second, for every variable T in the network a special subset (Markov Blanket) identifiable by the network is the minimal variable set required to predict T . However, all known algorithms learning a complete BN do not scale up beyond a few hundred variables. On the other hand, all known sound algorithms learning a local region of the network require an exponential number of training instances to the size of the learned region.The contribution of this paper is two-fold. We introduce a novel local algorithm that returns all variables with direct edges to and from a target variable T as well as a local algorithm that returns the Markov Blanket of T . Both algorithms (i) are sound, (ii) can be run efficiently in datasets with thousands of variables, and (iii) significantly outperform in terms of approximating the true neighborhood previous state-of-the-art algorithms using only a fraction of the training size required by the existing methods. A fundamental difference between our approach and existing ones is that the required sample depends on the generating graph connectivity and not the size of the local region; this yields up to exponential savings in sample relative to previously known algorithms. The results presented here are promising not only for discovery of local causal structure, and variable selection for classification, but also for the induction of complete BNs.

The max-min hill-climbing Bayesian network structure learning algorithm

“…The Bayesian network learning algorithm presented in this paper is based on the local discovery algorithm called Max-Min Parents and Children (MMPC) (a version of MMPC was published in Tsamardinos, Aliferis and Statnikov, 2003c). The Max-Min part of the algorithm name refers to the heuristic the algorithm uses, while the parents and children part refers to its output.…”

“…If G, P and G , P are two faithful Bayesian networks (to the same distribution), then for any variable T , it is the case that PC (Verma & Pearl, 1990;Pearl & Verma, 1991;Tsamardinos, Aliferis & Statnikov, 2003c). Thus, the set of parents and children of T is unique among all Bayesian networks faithful to the same distribution and so we will drop the superscript and denote it simply as PC T .…”

“…For example, in bioinformatics learning Bayesian networks have been used for the interpretation and discovery of gene regulatory pathways (Friedman et al, 2000). In addition, the theory of learning Bayesian networks has deep connections with variable selection for classification and has been used to design algorithms that optimally solve the problem under certain conditions Tsamardinos, Aliferis & Statnikov, 2003c). Finally, Bayesian network learning has been used in information retrieval (Baeze-Yates & Ribiero-Neto, 1999), natural language processing (Chapman et al, 2001), and for the analysis of a medical service's performance for management decisions (Acid et al, 2004).…”

Exploring Causal Relationships with Streaming Features

“…Two major algorithms HITON_PC and MMPC for the discovery of PC(T ) were introduced by Aliferis et al [37] and Tsamardinos et al [38], respectively. The Max-Min Parents and Children (MMPC) algorithm discovers the parents or children of a target of interest using a two-phase scheme.…”

Swamping and masking in Markov boundary discovery

“…This strategy is also used by divide-and-conquer search techniques, such as the max-min Markov boundary algorithm (Tsamardinos et al 2003b), the parents and children based Markov boundary (PCMB) algorithm (Peña et al 2007), the breadth first search of Markov boundary algorithm introduced by (Fu and Desmarais 2007), and the algorithms included in the algorithmic framework called GLL (Aliferis et al 2010a).…”