Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood

Gámez, José A.; Mateo, Juan L.; Puerta, José M.

doi:10.1007/s10618-010-0178-6

Cited by 174 publications

(103 citation statements)

References 31 publications

Supporting

Mentioning

101

Contrasting

Unclassified

Order By: Relevance

“…To learn the structure of PTs, we use the methods proposed above in combination with the two iterations constrained hillclimbing (2iCHC) algorithm [19]. 2iCHC is a version of the hill-climbing (HC) algorithm that uses a forbidden parents list to constrain the search space during the learning process, reducing the learning time of HC while assuring the return of a minimal I-map.…”

Section: Theorem 1 Let V Be a Sound Pt With Respect To A Bn B And A mentioning

confidence: 99%

Learning Bayesian networks with low inference complexity

2015

View full text Add to dashboard Cite

One of the main research topics in machine learning nowadays is the improvement of the inference and learning processes in probabilistic graphical models. Traditionally, inference and learning have been treated separately, but given that the structure of the model conditions the inference complexity, most learning methods will sometimes produce inefficient inference models. In this paper we propose a framework for learning low inference complexity Bayesian networks. For that, we use a representation of the network factorization that allows efficiently evaluating an upper bound in the inference complexity of each model during the learning process. Experimental results show that the proposed methods obtain tractable models that improve the accuracy of the predictions provided by approximate inference in models obtained with a well-known Bayesian network learner.

show abstract

Section: Theorem 1 Let V Be a Sound Pt With Respect To A Bn B And A mentioning

confidence: 99%

Learning Bayesian networks with low inference complexity

2015

View full text Add to dashboard Cite

show abstract

“…It is not straightforward to determine the BN that best reflects a particular problem from a database of cases because of the large number of possible DAG structures, given even a small number of nodes to connect (Daly et al 2011;Gamez et al 2011). Consequently, there have been a number of reports of heuristic search techniques to identify good models (Gamez et al 2011;Cooper and Herskovits 1992).…”

Section: Introductionmentioning

confidence: 99%

“…Consequently, there have been a number of reports of heuristic search techniques to identify good models (Gamez et al 2011;Cooper and Herskovits 1992). Recently, methods to design BN structures using evolutionary algorithms have appeared (Larranaga et al 2013); however, these have mostly used only the fittest solution in the final generation (Larranaga et al 1996a, b;Wong et al 1999).…”

Section: Introductionmentioning

confidence: 99%

Ensemble bayesian networks evolved with speciation for high-performance prediction in data mining

Kim

Cho

2015

Soft Comput

View full text Add to dashboard Cite

Bayesian networks (BNs) can be easily refined (or learn) using data given prior knowledge about a changing environment. Furthermore, by exploring multiple diverse BNs in parallel, it is expected that an intelligent system may adapt quickly to changes in the environment, resulting in robust prediction. Recently, there have been attempts to design BN structures using evolutionary algorithms; however, most of these have used only the fittest solution from the final generation. Because it is difficult to combine all of the important factors into a single evaluation function, the solution is often biased and of limited adaptability. Here we describe a method of generating diverse BN structures via speciation and selective combination for adaptive prediction. Experiments using the seven benchmark networks show that the proposed method can result in improved accuracy in handling uncertainty by exploiting ensembles of BNs evolved by speciation.

show abstract

“…Some recent studies have tried to improve this algorithm by constraining the search in order to obtain more scalable algorithms. The Constrained Hill-Climbing algorithm (CHC) and the iterated CHC algorithm (iCHC) (Gámez et al 2010) are based on the restriction of the candidate solutions to be evaluated during the search process when an independency between variables is found.…”

Section: Introductionmentioning

confidence: 99%

“…These algorithms achieve a significative reduction in computational demand with respect to the standard HC algorithm and are scalable to higher dimensionality domains. However, in Gámez et al (2010), it is proved that these algorithms are statistically worse than the HC algorithm in terms of accuracy.…”

Section: Introductionmentioning

confidence: 99%

Structural learning of Bayesian networks using local algorithms based on the space of orderings

2010

View full text Add to dashboard Cite

Structural learning of Bayesian networks (BNs)is an NP-hard problem which is generally addressed by means of heuristic search algorithms. Despite the fact that earlier proposals for dealing with this task were based on searching the space of Directed Acyclic Graphs (DAGs), there are some alternative approaches. One of these approaches for structural learning consists of searching the space of orderings, as given a certain topological order among the problem variables, it is relatively easy to build (and evaluate) a BN compatible with it. In practice, the latter methods make it possible to obtain good results, but they are still costly in terms of computation. In this article, we prove the correctness of the method used to evaluate each ordering, and we propose some efficient learning algorithms based on it. Our first proposal is based on the Hill-Climbing algorithm, and uses an improved neighbourhood definition. The second algorithm is an extension of the first one, and is based on the well-known Variable Neighbourhood Search metaheuristic. Finally, iterative versions of both algorithms are also proposed. The algorithms have been tested over a set of different domains, and have been compared with other methods such as HillClimbing in the space of DAGs or Greedy Equivalent Search, in order to study their behaviour in practice.

show abstract

Learning Bayesian networks by hill climbing: efficient methods based on progressive restriction of the neighborhood

Cited by 174 publications

References 31 publications

Learning Bayesian networks with low inference complexity

Learning Bayesian networks with low inference complexity

Ensemble bayesian networks evolved with speciation for high-performance prediction in data mining

Structural learning of Bayesian networks using local algorithms based on the space of orderings

Contact Info

Product

Resources

About