In search of the neural circuits of intrinsic motivation

Abstract

Children seem to acquire new know-how in a continuous and open-ended manner. In this paper, we hypothesize that an intrinsic motivation to progress in learning is at the origins of the remarkable structure of children's developmental trajectories. In this view, children engage in exploratory and playful activities for their own sake, not as steps toward other extrinsic goals. The central hypothesis of this paper is that intrinsically motivating activities correspond to expected decrease in prediction error. This motivation system pushes the infant to avoid both predictable and unpredictable situations in order to focus on the ones that are expected to maximize progress in learning. Based on a computational model and a series of robotic experiments, we show how this principle can lead to organized sequences of behavior of increasing complexity characteristic of several behavioral and developmental patterns observed in humans. We then discuss the putative circuitry underlying such an intrinsic motivation system in the brain and formulate two novel hypotheses. The first one is that tonic dopamine acts as a learning progress signal. The second is that this progress signal is directly computed through a hierarchy of microcortical circuits that act both as prediction and metaprediction systems.

Influence of reward motivation on human declarative memory

“…In more detail, while phasic dopamine increases originating from the VTA indicate cue-triggered response to reward availability (Schultz, 2010(Schultz, , 2007, tonic dopamine levels (minutes to hours), which for example increase during the exploration of novel stimuli, have been suggested to play a role in response vigor (Niv et al, 2007) and overall level of motivation (Ikemoto, 2007;Kaplan and Oudeyer, 2007). Similarly, a delay of reward by several seconds induces a prolongation of both prefrontal dopamine release and reward expectation-related activity in orbitofrontal and striatal neurons (Schultz, 2007), which may be another mechanism for extending the anticipatory excitement in time.…”

Curiosity-driven phonetic learning

“…Such intrinsically motivated exploration was also shown to generate automatically behavioural and cognitive developmental structures sharing interesting similarities with infant development [6], [12], [13]. This approach is grounded in psychological theories of intrinsic motivations [7], [14], explores several fundamental questions about curiosity-driven open-ended learning in robots [6], and allows to generate some novel hypotheses for the explanation of infant development, regarding behavioural [13], cognitive [12] and brain circuitry [15].…”

From “Oh, OK” to “Ah, yes” to “Aha!”: Hyper-systemizing and the rewards of insight

“…Consistent with the role of ACC as an affect-mediating task-progress monitor, processes in which actual-state to goal-state conflicts are small are experienced as pleasurable and motivating, presumably via activation of dopamine-driven reward pathways (Kaplan and Oudeyer, 2007). Hedonic experience is generated by activation of opiate and cannabinoid receptors in NAc and ventral pallidum, but is experienced as associated with sensory or imaginative input and influences attentional focus and motivation toward goals via its representation in OFC and ACC (Kringelbach and Berridge, 2009;Smith et al, 2009).…”

“…The pleasure associated with discovery and understanding is regarded as an intrinsic motivation toward learning, not only in academic environments (Gottfried, 1985), but also in curiosity-driven unstructured play (Gibson, 1988;Karmaloff-Smith, 1995;Kaplan and Oudeyer, 2007). However, common experience also suggests that there are large individual differences in the extent to which the pleasure of learning is motivational, and desires that do not have pleasurable experience as primary components are often motivational in both academic (Covington, 2000) and general (Reiss, 2004a) environments.…”