We examined the neural basis of self-regulation in individuals from a cohort of preschoolers who performed the delay-ofgratification task 4 decades ago. Nearly 60 individuals, now in their mid-forties, were tested on "hot" and "cool" versions of a go/ nogo task to assess whether delay of gratification in childhood predicts impulse control abilities and sensitivity to alluring cues (happy faces). Individuals who were less able to delay gratification in preschool and consistently showed low self-control abilities in their twenties and thirties performed more poorly than did high delayers when having to suppress a response to a happy face but not to a neutral or fearful face. This finding suggests that sensitivity to environmental hot cues plays a significant role in individuals' ability to suppress actions toward such stimuli. A subset of these participants (n = 26) underwent functional imaging for the first time to test for biased recruitment of frontostriatal circuitry when required to suppress responses to alluring cues. Whereas the prefrontal cortex differentiated between nogo and go trials to a greater extent in high delayers, the ventral striatum showed exaggerated recruitment in low delayers. Thus, resistance to temptation as measured originally by the delay-of-gratification task is a relatively stable individual difference that predicts reliable biases in frontostriatal circuitries that integrate motivational and control processes.reward | behavioral suppression | functional MRI | inferior frontal gyrus | longitudinal T he ability to resist temptation in favor of long-term goals is an essential component of individual, societal, and economical success. Developmentally, this ability has been assessed by measuring how long a young child can resist an immediate reward (e.g., a cookie) in favor of a larger, later reward (e.g., two cookies) (1). Even as adults we vary in our ability to resist temptations. Alluring situations can diminish our control (2-4); what serves as an alluring situation that requires a capacity to control our impulses, however, changes as a function of age (e.g., from cookies to social acceptance). In the present study we examined the extent to which individual differences in delay of gratification assessed when participants were in preschool and in their 20s and 30s predict control over impulses and sensitivity to social cues at the behavioral and neural level when the participants were in their 40s.Delay of gratification depends importantly on cognitive control (5). Cognitive control refers to the ability to suppress competing inappropriate thoughts or actions in favor of appropriate ones (6-11). Previously, we have shown that performance on the delay-of-gratification task in childhood predicts the efficiency with which the same individuals perform a cognitive control task (the go/nogo task) as adolescents and young adults (5). Individuals who as preschoolers directed their attention toward rewarding aspects of the classic delay-of-gratification situation, such as focusing on the cookie...

Humans are remarkably adept at generalizing knowledge between experiences in a way that can be difficult for computers. Often, this entails generalizing constituent pieces of experiences that do not fully overlap, but nonetheless share useful similarities with, previously acquired knowledge. However, it is often unclear how knowledge gained in one context should generalize to another. Previous computational models and data suggest that rather than learning about each individual context, humans build latent abstract structures and learn to link these structures to arbitrary contexts, facilitating generalization. In these models, task structures that are more popular across contexts are more likely to be revisited in new contexts. However, these models can only re-use policies as a whole and are unable to transfer knowledge about the transition structure of the environment even if only the goal has changed (or vice-versa). This contrasts with ecological settings, where some aspects of task structure, such as the transition function, will be shared between context separately from other aspects, such as the reward function. Here, we develop a novel non-parametric Bayesian agent that forms independent latent clusters for transition and reward functions, affording separable transfer of their constituent parts across contexts. We show that the relative performance of this agent compared to an agent that jointly clusters reward and transition functions depends environmental task statistics: the mutual information between transition and reward functions and the stochasticity of the observations. We formalize our analysis through an information theoretic account of the priors, and propose a meta learning agent that dynamically arbitrates between strategies across task domains to optimize a statistical tradeoff.

Humans spontaneously organize a continuous experience into discrete events and use the learned structure of these events to generalize and organize memory. We introduce the Structured Event Memory (SEM) model of event cognition, which accounts for human abilities in event segmentation, memory, and generalization. SEM is derived from a probabilistic generative model of event dynamics defined over structured symbolic scenes. By embedding symbolic scene representations in a vector space and parametrizing the scene dynamics in this continuous space, SEM combines the advantages of structured and neural network approaches to high-level cognition. Using probabilistic reasoning over this generative model, SEM can infer event boundaries, learn event schemata, and use event knowledge to reconstruct past experience. We show that SEM can scale up to high-dimensional input spaces, producing human-like event segmentation for naturalistic video data, and accounts for a wide array of memory phenomena.

We are grateful to Momchil Tomov for comments on an earlier draft of the paper.

Convergent evidence suggests that the basal ganglia support reinforcement learning by adjusting action values according to reward prediction errors. However, adaptive behavior in stochastic environments requires the consideration of uncertainty to dynamically adjust the learning rate. We consider how cholinergic tonically active interneurons (TANs) may endow the striatum with such a mechanism in computational models spanning three Marr's levels of analysis. In the neural model, TANs modulate the excitability of spiny neurons, their population response to reinforcement, and hence the effective learning rate. Long TAN pauses facilitated robustness to spurious outcomes by increasing divergence in synaptic weights between neurons coding for alternative action values, whereas short TAN pauses facilitated stochastic behavior but increased responsiveness to change-points in outcome contingencies. A feedback control system allowed TAN pauses to be dynamically modulated by uncertainty across the spiny neuron population, allowing the system to self-tune and optimize performance across stochastic environments.DOI: http://dx.doi.org/10.7554/eLife.12029.001