Motivational control of goal-directed action

Abstract

The control of goal-directed, instrumental actions by primary motivational states, such as hunger and thirst, is mediated by two processes . The first is engaged by the Pavlovian association between contextual or discriminative stimuli and the outcome or reinforcer presented during instrumental training . Such stimuli exert a motivational influence on instrumental performance that depends upon the relevance of the associated outcome to the current motivational state of the agent . Moreover, the motivational effects of these stimuli operate in the absence of prior experience with the outcome under the relevant motivational state . The second, instrumental, process is mediated by knowledge of the contingency between the action and its outcome and controls the value assigned to this outcome . In contrast to the Pavlovian process, motivational states do not influence the instrumental process directly ; rather, the agent has to learn about the value of an outcome in a given motivational state by exposure to it while in that state . This incentive learning is similar in certain respects to the acquisition of "cathexes" envisaged by Tolman (1949aTolman ( , 1949b .The general adaptive significance of the capacity for goal-directed action is so obvious as to require little or no comment. It is this capacity that allows us and other animals to control our environment in the service of our desires and needs. And yet, if asked the most simple questions about this capacity, such as why does an animal perform an instrumental action for a food reward more readily when hungry rather than sated, a contemporary psychologist could tell us little more than Hull (1943) or Tolman (1949aTolman ( , 1949b nearly half a century ago. The fact is that the study of the psychological processes controlling the performance of simple, goal-directed, instrumental actions by basic primary motivational states has been neglected over the intervening decades.It is true that we know much more about the neurophysiological and chemical mechanisms regulating consummatory behavior, such as eating, drinking, and copulating, and indeed about the major role of learning in such behaviors. A contemporary psychologist could also tell us much, at least on a functional level, about how instrumental behavior adapts to schedule and foraging constraints, and indeed about the effects of motivational variables on such adaptation . Moreover, there is an extensive literature on the psychological resources brought to bear in solving complex problems and decisions in the service of cognitive goals . But even so, we should search in vain among the literature for a consensus about the psychological processes by which primary motivational states, such as hunger and thirst, regulate simple goal-directed acts . This is the issue that we wish to address in the present paper. GOAL-DIRECTED ACTIONA prerequisite for this discussion is a clear specification of what we mean by "goal directed . " Our conception of a goal-directed action is psychological, being based upon the nature ...

Modeling functions of striatal dopamine modulation in learning and planning

“…Thus, the model improves corticostriatal weights by simulating experience with the Extended TD model (Sutton and Barto, 1981;Sutton and Barto, 1998). Consistent with this simulation result, motor performance for human and animals can be improved by stimulating planning processes (Dickinson and Balleine, 1994;Decety, 1996).…”

“…Animals learn act outcomes and select their acts based on the motivational value of these outcomes (reviews in Thistlethwaite, 1951;MacCorquodale and Meehl, 1954;Mackintosh 1974;Dennett 1978;Dickinson 1980;Dickinson 1994;Dickinson and Balleine 1994;Balleine and Dickinson 1998). Such goal-directed behavior is termed "planning" in reinforcement learning studies, or "cognition" in animal learning studies (Craik 1943;Sutton andBarto, 1981, 1998;Sutton and Pinette 1985;Dickinson 1994). Planning and sensorimotor learning were demonstrated for rats in T-maze experiments ( Fig.…”

“…Furthermore, animals seem to use these predictions to select their acts (Sutton and Pinette, 1985;Sutton and Barto 1981;Dickinson and Balleine 1994;Balleine and Dickinson 1998). This insight led to the implementation of neural network architectures in which an internal model, serving as the Critic, transiently influences the Actor to elicit acts (Sutton and Barto, 1981).…”

The role of learning-related dopamine signals in addiction vulnerability

“…These three features are dissociable, but rely on partially overlapping neural mechanisms . Incentive salience in this context is distinct from incentive motivational properties or "incentive value" in instrumental settings as defined by Dickinson and colleagues (Dickinson and Balleine, 1994;Dickinson et al, 2000).…”

Feelings of control: Contingency determines experience of action

“…Such operant actions imply the ability to know that one is performing an action, and to represent its consequences (Dickinson & Balleine, 2000). Research on the epistemic content and conscious experience of action has identified two distinct processes underlying this ability.…”