A theoretical analysis of the BDeu scores in Bayesian network structure learning

Suzuki, Joe

doi:10.1007/s41237-016-0006-4

Cited by 27 publications

(36 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is typically used with small imaginary sample sizes such as α i = 1 as suggested by [2] and [41]. Alternative BD scores have been proposed in [42] and [43,44].…”

Section: Statistical Criteria: Conditional Independence Tests and Netmentioning

confidence: 99%

Who learns better Bayesian network structures: Accuracy and speed of structure learning algorithms

Scutari

Graafland

Gutiérrez

2019

International Journal of Approximate Reasoning

200

147

View full text Add to dashboard Cite

Three classes of algorithms to learn the structure of Bayesian networks from data are common in the literature: constraint-based algorithms, which use conditional independence tests to learn the dependence structure of the data; score-based algorithms, which use goodness-of-fit scores as objective functions to maximise; and hybrid algorithms that combine both approaches. Constraint-based and score-based algorithms have been shown to learn the same structures when conditional independence and goodness of fit are both assessed using entropy and the topological ordering of the network is known [1].In this paper, we investigate how these three classes of algorithms perform outside the assumptions above in terms of speed and accuracy of network reconstruction for both discrete and Gaussian Bayesian networks. We approach this question by recognising that structure learning is defined by the combination of a statistical criterion and an algorithm that determines how the criterion is applied to the data. Removing the confounding effect of different choices for the statistical criterion, we find using both simulated and realworld complex data that constraint-based algorithms are often less accurate than score-based algorithms, but are seldom faster (even at large sample sizes); and that hybrid algorithms are neither faster nor more accurate than constraint-based algorithms. This suggests that commonly held beliefs on structure learning in the literature are strongly influenced by the choice of particular statistical criteria rather than just by the properties of the algorithms themselves.

show abstract

“…It is typically used with small imaginary sample sizes such as α i = 1 as suggested by [2] and [41]. Alternative BD scores have been proposed in [42] and [43,44].…”

Section: Statistical Criteria: Conditional Independence Tests and Netmentioning

confidence: 99%

Who learns better Bayesian network structures: Accuracy and speed of structure learning algorithms

Scutari

Graafland

Gutiérrez

2019

International Journal of Approximate Reasoning

200

147

View full text Add to dashboard Cite

show abstract

“…for α ijk = 1 we obtain the K2 score from Cooper and Herskovits (1991); -for α ijk = 1 /2 we obtain the BD score with Jeffrey's prior (BDJ; Suzuki, 2016); -for α ijk = α/(r i q i ) we obtain the BDeu score from Heckerman et al (1995), which is the most common choice in the BD family and has α i = α for all X i ; -for α ijk = α/(r iqi ), whereq i is the number of Π G Xi such that n ij > 0, we obtain the BD sparse (BDs) score recently proposed in Scutari (2016);…”

Section: Bayesian Dirichlet Marginal Likelihoodsmentioning

confidence: 99%

“…Finally, Suzuki (2016) studied the asymptotic properties of BDeu by contrasting it with BDJ. He found that BDeu is not regular in the sense that it may learn DAGs in a way that is not consistent with either the MDL principle (through BIC) or the ranking of those DAGs given by their entropy.…”

Section: Bdeu and Bayesian Model Selectionmentioning

confidence: 99%

See 1 more Smart Citation

Dirichlet Bayesian network scores and the maximum relative entropy principle

Scutari

2018

Behaviormetrika

View full text Add to dashboard Cite

A classic approach for learning Bayesian networks from data is to identify a maximum a posteriori (MAP) network structure. In the case of discrete Bayesian networks, MAP networks are selected by maximising one of several possible Bayesian Dirichlet (BD) scores; the most famous is the Bayesian Dirichlet equivalent uniform (BDeu) score from Heckerman et al (1995). The key properties of BDeu arise from its uniform prior over the parameters of each local distribution in the network, which makes structure learning computationally efficient; it does not require the elicitation of prior knowledge from experts; and it satisfies score equivalence.In this paper we will review the derivation and the properties of BD scores, and of BDeu in particular, and we will link them to the corresponding entropy estimates to study them from an information theoretic perspective. To this end, we will work in the context of the foundational work of Giffin and Caticha (2007), who showed that Bayesian inference can be framed as a particular case of the maximum relative entropy principle. We will use this connection to show that BDeu should not be used for structure learning from sparse data, since it violates the maximum relative entropy principle; and that it is also problematic from a more classic Bayesian model selection perspective, because it produces Bayes factors that are sensitive to the value of its only hyperparameter. Using a large simulation study, we found in our previous work (Scutari, 2016) that the Bayesian Dirichlet sparse (BDs) score seems to provide better accuracy in structure learning; in this paper we further show that BDs does not suffer from the issues above, and we recommend to use it for sparse data instead of BDeu. Finally, will show that these issues are in fact different aspects of the same problem and a consequence of the distributional assumptions of the prior.

show abstract

“…, and A × B, respectively; the computation for {0, 1} n can be extended to those for A n , B n , and (A × B) n , respectively. For example, for A = {0, 1, · · · , α − 1} with α = 2, constants a, b and occurrences c, n − c are replaced by a(x) and c(x), respectively, for x = 0, 1, · · · , α − 1; thus, the extended formula can be expressed by [7], [14] Q…”

Section: Forest Learning From Complete Datamentioning

confidence: 99%

Forest Learning From Data and its Universal Coding

Suzuki

2018

IEEE Trans. Inform. Theory

Self Cite

View full text Add to dashboard Cite

This paper considers structure learning from data with n samples of p variables, assuming that the structure is a forest, using the Chow-Liu algorithm. Specifically, for incomplete data, we construct two model selection algorithms that complete in O(p 2 ) steps: one obtains a forest with the maximum posterior probability given the data, and the other obtains a forest that converges to the true one as n increases. We show that the two forests are generally different when some values are missing. Additionally, we present estimations for benchmark data sets to demonstrate that both algorithms work in realistic situations. Moreover, we derive the conditional entropy provided that no value is missing, and we evaluate the per-sample expected redundancy for the universal coding of incomplete data in terms of the number of non-missing samples.

show abstract

A theoretical analysis of the BDeu scores in Bayesian network structure learning

Cited by 27 publications

References 9 publications

Who learns better Bayesian network structures: Accuracy and speed of structure learning algorithms

Who learns better Bayesian network structures: Accuracy and speed of structure learning algorithms

Dirichlet Bayesian network scores and the maximum relative entropy principle

Forest Learning From Data and its Universal Coding

Contact Info

Product

Resources

About