Markov Chains

Norris, James R.

doi:10.1017/cbo9780511810633

Cited by 1,558 publications

(1,423 citation statements)

References 0 publications

Supporting

Mentioning

1,391

Contrasting

Unclassified

Order By: Relevance

“…A Markov chain is a sequence of random variables where the value taken by each variable (known as the state of the Markov chain) depends only on the value taken by the previous variable (Norris, 1997). We can generate a sequence of states from a Markov chain by iteratively drawing each variable from its distribution conditioned on the previous variable.…”

Section: Markov Chain Monte Carlomentioning

confidence: 99%

Uncovering mental representations with Markov chain Monte Carlo

Sanborn

Griffiths

Shiffrin

2010

Cognitive Psychology

View full text Add to dashboard Cite

A note on versions:The version presented here may differ from the published version or, version of record, if you wish to cite this item you are advised to consult the publisher's version. Please see the 'permanent WRAP URL' above for details on accessing the published version and note that access may require a subscription. Richard M. Shiffrin Indiana University, Bloomington Abstract A key challenge for cognitive psychology is the investigation of mental representations, such as object categories, subjective probabilities, choice utilities, and memory traces. In many cases, these representations can be expressed as a non-negative function defined over a set of objects. We present a behavioral method for estimating these functions. Our approach uses people as components of a Markov chain Monte Carlo (MCMC) algorithm, a sophisticated sampling method originally developed in statistical physics. Experiments 1 and 2 verified the MCMC method by training participants on various category structures and then recovering those structures. Experiment 3 demonstrated that the MCMC method can be used estimate the structures of the real-world animal shape categories of giraffes, horses, dogs, and cats. Experiment 4 combined the MCMC method with multidimensional scaling to demonstrate how different accounts of the structure of categories, such as prototype and exemplar models, can be tested, producing samples from the categories of apples, oranges, and grapes.

show abstract

Section: Markov Chain Monte Carlomentioning

confidence: 99%

Uncovering mental representations with Markov chain Monte Carlo

Sanborn

Griffiths

Shiffrin

2010

Cognitive Psychology

View full text Add to dashboard Cite

show abstract

“…One is based on a classical, i.e. non-probabilistic, reduction relation on probability distributions on classical terms; a second one is defined via a probabilistic transition relation on probabilistic λ-terms which subsumes the classical β-reduction; and a third one is a linear operator semantics which specifies the computational dynamics in the form of so-called Markov chains [20]. All the three semantics are equivalent in the sense that they assign the same meaning to a probabilistic λ-term P .…”

Section: Resultsmentioning

confidence: 99%

Probabilistic -calculus and Quantitative Program Analysis

Pierro

Hankin

Wiklicky

2005

Journal of Logic and Computation

View full text Add to dashboard Cite

We show how the framework of probabilistic abstract interpretation can be applied to statically analyse a probabilistic version of the λ-calculus. The resulting analysis allows for a more speculative use of its outcomes based on the consideration of statistically defined quantities. After introducing a linear operator based semantics for our probabilistic λ-calculus Λp, and reviewing the framework of abstract interpretation and strictness analysis, we demonstrate our technique by constructing a probabilistic (first-order) strictness analysis for Λp.

show abstract

“…A distribution π is said to be a limiting distribution of matrix P if there exists a distribution µ such that µ · P n → π as n → ∞; if P has exactly one limiting distribution then it is said to have a unique limiting distribution. If matrix P has a unique limiting distribution π then it is also a unique stationary distribution but not vice versa [18]. Fix a set S = {1, 2, .…”

Section: B Stochastic Matricesmentioning

confidence: 99%

Reasoning about MDPs as Transformers of Probability Distributions

Korthikanti

Viswanathan

Agha

et al. 2010

2010 Seventh International Conference on the Quantitative Evaluation of Systems

View full text Add to dashboard Cite

Abstract-We consider Markov Decision Processes (MDPs) as transformers on probability distributions, where with respect to a scheduler that resolves nondeterminism, the MDP can be seen as exhibiting a behavior that is a sequence of probability distributions. Defining propositions using linear inequalities, one can reason about executions over the space of probability distributions. In this framework, one can analyze properties that cannot be expressed in logics like PCTL * , such as expressing bounds on transient rewards and expected values of random variables, and comparing the probability of being in one set of states at a given time with that of being in another set of states. We show that model checking MDPs with this semantics against ω-regular properties is in general undecidable. We then identify special classes of propositions and schedulers with respect to which the model checking problem becomes decidable. We demonstrate the potential usefulness of our results through an example.

show abstract

Markov Chains

Cited by 1,558 publications

References 0 publications

Uncovering mental representations with Markov chain Monte Carlo

Uncovering mental representations with Markov chain Monte Carlo

Probabilistic -calculus and Quantitative Program Analysis

Reasoning about MDPs as Transformers of Probability Distributions

Contact Info

Product

Resources

About