The computational pharmacology of oculomotion

Parr, Thomas; Friston, Karl J.

doi:10.1007/s00213-019-05240-0

Cited by 18 publications

(12 citation statements)

References 81 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that the set-up described here takes place in the categorical domain. Similarly, computational models of working memory tasks are sometimes formulated in terms of categorical partially observed Markov decision processes [ 54 , 55 , 56 ]. This underwrites the importance of demonstrating the emergence of conditional dependencies in Markov blanketed categorical systems.…”

Section: Discussionmentioning

confidence: 99%

Memory and Markov Blankets

Parr

Costa

Heins

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

In theoretical biology, we are often interested in random dynamical systems—like the brain—that appear to model their environments. This can be formalized by appealing to the existence of a (possibly non-equilibrium) steady state, whose density preserves a conditional independence between a biological entity and its surroundings. From this perspective, the conditioning set, or Markov blanket, induces a form of vicarious synchrony between creature and world—as if one were modelling the other. However, this results in an apparent paradox. If all conditional dependencies between a system and its surroundings depend upon the blanket, how do we account for the mnemonic capacity of living systems? It might appear that any shared dependence upon past blanket states violates the independence condition, as the variables on either side of the blanket now share information not available from the current blanket state. This paper aims to resolve this paradox, and to demonstrate that conditional independence does not preclude memory. Our argument rests upon drawing a distinction between the dependencies implied by a steady state density, and the density dynamics of the system conditioned upon its configuration at a previous time. The interesting question then becomes: What determines the length of time required for a stochastic system to ‘forget’ its initial conditions? We explore this question for an example system, whose steady state density possesses a Markov blanket, through simple numerical analyses. We conclude with a discussion of the relevance for memory in cognitive systems like us.

show abstract

Section: Discussionmentioning

confidence: 99%

Memory and Markov Blankets

Parr

Costa

Heins

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…This discussion suggests that from a generic generative model, we can derive Bayesian updates that clarify how perception, policy selection, and actions shape beliefs about hidden states and subsequent outcomes in a dynamic (nonstationary) environment. This formulation can be extended to capture a more representative generative process by defining a hierarchical (deep temporal) generative model (described in Friston, FitzGerald, et al, 2017;, continuous statespace models (Buckley, Kim, McGregor, & Seth, 2017;Parr & Friston, 2019a;Ueltzhöffer, 2018) or mixed models with both discrete and continuous states as described in and Parr and Friston (2018). In the case of a continuous formulation, the generative model state-space can be defined in terms of generalized coordinates of motion (i.e., the coefficients of a Taylor series expansion in time, as opposed to a series of discrete time steps), which generally have a nonlinear mapping to the observed outcomes.…”

Section: Optimizing Free Energymentioning

confidence: 99%

Active Inference: Demystified and Compared

et al. 2021

Self Cite

View full text Add to dashboard Cite

Active inference is a first principle account of how autonomous agents operate in dynamic, nonstationary environments. This problem is also considered in reinforcement learning, but limited work exists on comparing the two approaches on the same discrete-state environments. In this letter, we provide (1) an accessible overview of the discrete-state formulation of active inference, highlighting natural behaviors in active inference that are generally engineered in reinforcement learning, and (2) an explicit discrete-state comparison between active inference and reinforcement learning on an OpenAI gym baseline. We begin by providing a condensed overview of the active inference literature, in particular viewing the various natural behaviors of active inference agents through the lens of reinforcement learning. We show that by operating in a pure belief-based setting, active inference agents can carry out epistemic exploration—and account for uncertainty about their environment—in a Bayes-optimal fashion. Furthermore, we show that the reliance on an explicit reward signal in reinforcement learning is removed in active inference, where reward can simply be treated as another observation we have a preference over; even in the total absence of rewards, agent behaviors are learned through preference learning. We make these properties explicit by showing two scenarios in which active inference agents can infer behaviors in reward-free environments compared to both Q-learning and Bayesian model-based reinforcement learning agents and by placing zero prior preferences over rewards and learning the prior preferences over the observations corresponding to reward. We conclude by noting that this formalism can be applied to more complex settings (e.g., robotic arm movement, Atari games) if appropriate generative models can be formulated. In short, we aim to demystify the behavior of active inference agents by presenting an accessible discrete state-space and time formulation and demonstrate these behaviors in a OpenAI gym environment, alongside reinforcement learning agents.

show abstract

“…This explains why – in active inference – agent behaviour is often modelled using a discrete state-space formulation, the particular applications of which are summarised in Table 1 . More recently, mixed generative models ( Friston, Parr et al, 2017 ) – combining discrete and continuous states – have been used to model behaviour involving discrete and continuous representations (e.g., decision-making and movement ( Parr & Friston, 2018d ), speech production and recognition ( Friston, Sajid et al, 2020 ), pharmacologically induced changes in eye-movement control ( Parr & Friston, 2019 ) or reading; involving continuous visual sampling informing inferences about discrete semantics ( Friston, Parr et al, 2017 )).…”

Section: Introductionmentioning

confidence: 99%

“… Benrimoh et al, 2018 , Parr, Benrimoh et al, 2018 , Parr and Friston, 2018c , Parr, Rees et al, 2018 and Parr, Rikhye et al (2019) Neuromodulation Use of precision parameters to manipulate exploration during saccadic searches; associating uncertainty with cholinergic and noradrenergic systems. Parr and Friston, 2017a , Parr and Friston, 2019 , Sales et al, 2018 and Vincent et al (2019) Decisions to movements Mixed generative models combining discrete and continuous states to implement decisions through movement. Friston, Parr et al (2017) and Parr and Friston (2018d) Planning, navigation and niche construction Agent induced changes in environment (generative process); decomposition of goals into subgoals.…”

Section: Introductionmentioning

confidence: 99%

Active inference on discrete state-spaces: A synthesis

Costa

Parr

Sajid

et al. 2020

Journal of Mathematical Psychology

Self Cite

205

203

View full text Add to dashboard Cite

Active inference is a normative principle underwriting perception, action, planning, decision-making and learning in biological or artificial agents. From its inception, its associated process theory has grown to incorporate complex generative models, enabling simulation of a wide range of complex behaviours. Due to successive developments in active inference, it is often difficult to see how its underlying principle relates to process theories and practical implementation. In this paper, we try to bridge this gap by providing a complete mathematical synthesis of active inference on discrete state-space models. This technical summary provides an overview of the theory, derives neuronal dynamics from first principles and relates this dynamics to biological processes. Furthermore, this paper provides a fundamental building block needed to understand active inference for mixed generative models; allowing continuous sensations to inform discrete representations. This paper may be used as follows: to guide research towards outstanding challenges, a practical guide on how to implement active inference to simulate experimental behaviour, or a pointer towards various in-silico neurophysiological responses that may be used to make empirical predictions.

show abstract

The computational pharmacology of oculomotion

Cited by 18 publications

References 81 publications

Memory and Markov Blankets

Memory and Markov Blankets

Active Inference: Demystified and Compared

Active inference on discrete state-spaces: A synthesis

Contact Info

Product

Resources

About