Backward Simulation in Bayesian Networks

Fung, Robert; Favero, Brendan Del

doi:10.1016/b978-1-55860-332-5.50034-1

Cited by 36 publications

(28 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…A variety of Monte Carlo algorithms have been developed (see Neal, 1993) and applied to the inference problem in graphical models (Dagum & Luby, 1993;Fung & Favero, 1994;Gilks, Thomas, & Spiegelhalter, 1994;Jensen, Kong, & Kjaerulff, 1995;Pearl, 1988). Advantages of these algorithms include their simplicity of implementation and theoretical guarantees of convergence.…”

Section: P(h | E) = P(h E) P(e)mentioning

confidence: 99%

Untitled

Jordan

Ghahramani

Jaakkola³

et al. 1999

Machine Learning

2,147

175

View full text Add to dashboard Cite

Abstract. This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models (Bayesian networks and Markov random fields). We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, which exploit laws of large numbers to transform the original graphical model into a simplified graphical model in which inference is efficient. Inference in the simpified model provides bounds on probabilities of interest in the original model. We describe a general framework for generating variational transformations based on convex duality. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.

show abstract

Section: P(h | E) = P(h E) P(e)mentioning

confidence: 99%

Untitled

Jordan

Ghahramani

Jaakkola³

et al. 1999

Machine Learning

2,147

175

View full text Add to dashboard Cite

show abstract

“…A variety o f M o n te Carlo algorithms have been developed see MacKay, this volume, and Neal, 1993 and applied to the inference problem in graphical models Dagum & Luby, 1993;Fung & Favero, 1994;Gilks, Thomas, & Spiegelhalter, 1994;Jensen, Kong, & Kj rul , 1995;Pearl, 1988 . Advantages of these algorithms include their simplicity of implementation and theoretical guarantees of convergence.…”

Section: Introductionmentioning

confidence: 99%

An Introduction to Variational Methods for Graphical Models

Jordan

Ghahramani

Jaakkola

et al. 1998

Learning in Graphical Models

1,589

2,090

View full text Add to dashboard Cite

Abstract. This paper presents a tutorial introduction to the use of variational methods for inference and learning in graphical models. We present a number of examples of graphical models, including the QMR-DT database, the sigmoid belief network, the Boltzmann machine, and several variants of hidden Markov models, in which it is infeasible to run exact inference algorithms. We then introduce variational methods, showing how upper and lower bounds can be found for local probabilities, and discussing methods for extending these bounds to bounds on global probabilities of interest. Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.

show abstract

“…Randomly sample from Cat according to its prior probability (say, we choose False). There are a bewilderingly large variety of variants of this scheme, both in the uncertainty in AI literature (Fung 1994;Shachter 1989;Henrion 1988;Pearl 1987) and in the traditional statistical literature. All suffer from three problems: (1) the basic approximation problem (for example, the problem of determining if a probability is less than a specified value) is NP-hard (Dagum 1993), (2) error decreases as the square of the number of samples, and (3) unexpected evidence on nonroot nodes can reduce the number of useful samples collected.…”

Section: Approximate Inference Simulationmentioning

confidence: 99%

“…All suffer from three problems: (1) the basic approximation problem (for example, the problem of determining if a probability is less than a specified value) is NP-hard (Dagum 1993), (2) error decreases as the square of the number of samples, and (3) unexpected evidence on nonroot nodes can reduce the number of useful samples collected. The last is particularly problematic and has largely restricted the use of such methods to the computation of priors; see Fung (1994).…”

Section: Approximate Inference Simulationmentioning

confidence: 99%

Inference in Bayesian Networks

Vomlel¹

2003

Chapman &Amp; Hall/CRC Computer Science &Amp; Data Analysis

View full text Add to dashboard Cite

s A Bayesian network is a compact, expressive representation of uncertain relationships among parameters in a domain. In this article, I introduce basic methods for computing with Bayesian networks, starting with the simple idea of summing the probabilities of events of interest. The article introduces major current methods for exact computation, briefly surveys approximation methods, and closes with a brief discussion of open issues.I n rare cases, primarily involving terminological information or other artificially constructed domains, one has the opportunity to determine categorically the truth of a proposition based on prior knowledge and current observation. Often, truth is more elusive, and categorical statements can only be made by judgment of the likelihood or other ordinal attribute of competing propositions. Probability theory is the oldest and best-understood theory for representing and reasoning about such situations, but early AI experimental efforts at applying probability theory were disappointing and only confirmed a belief among AI researchers that those who worried about numbers were "missing the point." 1 The point, so succinctly stated in Newell and Simon's physical symbol system hypothesis, 2 was that structure was the key, not the numeric details. The problem: The core around which a probabilistic approach revolves is the joint-probability distribution (JPD). Unfortunately, for domains described by a set of discrete parameters, the size of this object and the complexity of reasoning with it directly can both be exponential in the number of parameters.A popular simplification was the naïve Bayes's model. This model assumes that the probability distribution for each observable parameter (that is, the probability of each value in the domain of the parameter) depends only on the root cause and not on the other parameters. Simplifying assumptions such as naïve Bayes's permitted tractable reasoning but were too extreme: They again provided no mechanism for representing the qualitative structure of a domain.About 10 years ago, probability, and especially decision theory, began to attract renewed interest within the AI community, which was the result of a felicitous combination of obstacle and opportunity: The issue of ordering possible beliefs, both for belief revision and for action selection, was seen as increasingly important and problematic, and at the same time, dramatic new developments in computational probability and decision theory directly addressed perceived shortcomings. The key development was the discovery that a relationship could be established between a welldefined notion of conditional independence in probability theory and the absence of arcs in a directed acyclic graph (DAG). This relationship made it possible to express much of the structural information in a domain independently of the detailed numeric information, in a way that both simplifies knowledge acquisition and reduces the computational complexity of reasoning. The resulting graphic models have come to be known as Ba...

show abstract

Backward Simulation in Bayesian Networks

Cited by 36 publications

References 12 publications

Untitled

Untitled

An Introduction to Variational Methods for Graphical Models

Inference in Bayesian Networks

Contact Info

Product

Resources

About