Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction

Akiti, Korleki; Tsutsui-Kimura, Iku; Xie, Yu; Mathis, Alexander; Markowitz, Jeffrey E.; Anyoha, Rockwell; Datta, Sandeep Robert; Mathis, Mackenzie Weygandt; Uchida, Naoshige; Watabe-Uchida, Mitsuko

doi:10.1016/j.neuron.2022.08.022

Cited by 55 publications

(63 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig 5 shows model fits on the 26 mice from Akiti et al (2022). The animal ranking is sorted firstly on animal group and secondly on total time spent near the object.…”

Section: Resultsmentioning

confidence: 99%

Risking your Tail: Modeling Individual Differences in Risk-sensitive Exploration using Bayes Adaptive Markov Decision Processes

Shen,

Dayan

2024

Preprint

View full text Add to dashboard Cite

Novelty is a double-edged sword for agents and animals alike: they might benefit from untapped resources or face unexpected costs or dangers such as predation. The conventional exploration/exploitation tradeoff is thus coloured by risk-sensitivity. A wealth of experiments has shown how animals solve this dilemma, for example using intermittent approach. However, there are large individual differences in the nature of approach, and modeling has yet to elucidate how this might be based on animals' differing prior expectations about reward and threat and degrees of risk aversion. To capture these factors, we built a Bayes adaptive Markov decision process model with three key components: an adaptive hazard function capturing potential predation, an intrinsic reward function providing the urge to explore, and a conditional value at risk (CVaR) objective, which is a contemporary measure of trait risk-sensitivity. We fit this model to a coarse-grain abstraction of the behaviour of 26 animals who freely explored a novel object in an open-field arena (Akiti et al.Neuron110, 2022). We show that the model captures both quantitative (frequency, duration of exploratory bouts) and qualitative (stereotyped tail-behind) features of behavior, including the substantial idiosyncrasies that were observed. We find that “brave” animals, though varied in their behavior, generally are more risk neutral, and enjoy a flexible hazard prior. They begin with cautious exploration, and quickly transition to confident approach to maximize exploration for reward. On the other hand, “timid” animals, characterized by risk aversion and high and inflexible hazard priors, display self-censoring that leads to the sort of asymptotic maladaptive behavior that is often associated with psychiatric illnesses such as anxiety and depression. Explaining risk-sensitive exploration using factorized parameters of reinforcement learning models could aid in the understanding, diagnosis, and treatment of psychiatric abnormalities in humans and other animals.

show abstract

“…Fig 5 shows model fits on the 26 mice from Akiti et al (2022). The animal ranking is sorted firstly on animal group and secondly on total time spent near the object.…”

Section: Resultsmentioning

confidence: 99%

Risking your Tail: Modeling Individual Differences in Risk-sensitive Exploration using Bayes Adaptive Markov Decision Processes

Shen,

Dayan

2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Such observations are consistent with multiple neuromodulators acting as prediction errors across the range of valence. For example, recent work has suggested that dopamine acts as a threat prediction error in addition to its role in reward, and that threat behavior can be best explained by reinforcement learning models which incorporate uncertainty (Akiti et al, 2022). While dopamine has also been proposed to encode salience across valence based on novelty induced dopamine release, we did not observe NE release to non-associated cues, suggesting a bias towards valence (Kutlu et al, 2021), we did not observe significant NE release to a novel, non-associated cue.…”

Section: Discussionmentioning

confidence: 99%

Prefrontal norepinephrine represents a threat prediction error under uncertainty

Basu

Yang

et al. 2022

Preprint

View full text Add to dashboard Cite

Animals must learn to predict constantly varying threats in the environment to survive by enacting defensive behaviors. Dopamine is involved in the prediction of rewards, encoding a reward prediction error in a similar manner to temporal difference learning algorithm. However, the corresponding molecular and computational form of threat prediction errors is not as well-characterized, although norepinephrine and other neuromodulators and neuropeptides participate in fear learning. Here, we utilized fluorescent norepinephrine recordings over the course of fear learning in concert with reinforcement learning modeling to identify its role in the prediction of threat. By varying timing and sensory uncertainty in the formation of threat associations, we were able to define a precise computational role for norepinephrine in this process. Norepinephrine release approximates the strength of fear associations, and its temporal dynamics are compatible with a prediction error signal. Intriguingly, the release of norepinephrine is influenced by time and sensory feedback, serving as an antithesis of the classical reward prediction error role of dopamine. Thus, these results directly demonstrate a combined cognitive and affective role of norepinephrine in the prediction of threat, with implications for neuropsychiatric disorders such as anxiety and PTSD.

show abstract

“…. , V camera sees a single 2D perspective projection of ȳ denoted as y(v) ∈ R 2 , in pixel coordinates 3 . Thus, we have a 2V -dimensional measurement y k (1) T • • • y k (V ) T of our 3D keypoint ȳk .…”

Section: Introductionmentioning

confidence: 99%

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

Biderman

Whiteway

Hurwitz

et al. 2023

Preprint

View full text Add to dashboard Cite

Pose estimation algorithms are shedding new light on animal behavior and intelligence. Most existing models are only trained with labeled frames (supervised learning). Although effective in many cases, the fully supervised approach requires extensive image labeling, struggles to generalize to new videos, and produces noisy outputs that hinder downstream analyses. We address each of these limitations with a semi-supervised approach that leverages the spatiotemporal statistics of unlabeled videos in two different ways. First, we introduce unsupervised training objectives that penalize the network whenever its predictions violate smoothness of physical motion, multiple-view geometry, or depart from a low-dimensional subspace of plausible body configurations. Second, we design a new network architecture that predicts pose for a given frame using temporal context from surrounding unlabeled frames. These context frames help resolve brief occlusions or ambiguities between nearby and similar-looking body parts. The resulting pose estimation networks achieve better performance with fewer labels, generalize better to unseen videos, and provide smoother and more reliable pose trajectories for downstream analysis; for example, these improved pose trajectories exhibit stronger correlations with neural activity. We also propose a Bayesian post-processing approach based on deep ensembling and Kalman smoothing that further improves tracking accuracy and robustness. We release a deep learning package that adheres to industry best practices, supporting easy model development and accelerated training and prediction. Our package is accompanied by a cloud application that allows users to annotate data, train networks, and predict new videos at scale, directly from the browser.

show abstract

Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction

Cited by 55 publications

References 76 publications

Risking your Tail: Modeling Individual Differences in Risk-sensitive Exploration using Bayes Adaptive Markov Decision Processes

Risking your Tail: Modeling Individual Differences in Risk-sensitive Exploration using Bayes Adaptive Markov Decision Processes

Prefrontal norepinephrine represents a threat prediction error under uncertainty

Lightning Pose: improved animal pose estimation via semi-supervised learning, Bayesian ensembling, and cloud-native open-source tools

Contact Info

Product

Resources

About