Intentional action: from anticipation to goal-directed behavior

Pezzulo, Giovanni; Castelfranchi, Cristiano

doi:10.1007/s00426-009-0241-3

Cited by 21 publications

(12 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It has been argued that optimal stopping and more sophisticated forms of meta-choice could be based on mechanisms for monitoring, uncertainty consideration and behavioral inhibition. With these mechanisms, the architectures for action specification and selection described before can become able of goal-directed choice and cognitive control, consistent with the view that these more advanced abilities could derive from elaborations of brain designs that solve simpler sensorimotor processes (Pezzulo, 2008, 2011; Pezzulo and Castelfranchi, 2009; Cisek and Kalaska, 2010; Cisek, 2012). In the MIC, these mechanisms could improve the choice by permitting model-based mechanisms to support or even substitute the default habitual control mode.…”

Section: Resultssupporting

confidence: 74%

The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation

Pezzulo¹,

Rigoli²,

Chersi³

2013

Front. Psychol.

Self Cite

152

207

View full text Add to dashboard Cite

Instrumental behavior depends on both goal-directed and habitual mechanisms of choice. Normative views cast these mechanisms in terms of model-free and model-based methods of reinforcement learning, respectively. An influential proposal hypothesizes that model-free and model-based mechanisms coexist and compete in the brain according to their relative uncertainty. In this paper we propose a novel view in which a single Mixed Instrumental Controller produces both goal-directed and habitual behavior by flexibly balancing and combining model-based and model-free computations. The Mixed Instrumental Controller performs a cost-benefits analysis to decide whether to chose an action immediately based on the available “cached” value of actions (linked to model-free mechanisms) or to improve value estimation by mentally simulating the expected outcome values (linked to model-based mechanisms). Since mental simulation entails cognitive effort and increases the reward delay, it is activated only when the associated “Value of Information” exceeds its costs. The model proposes a method to compute the Value of Information, based on the uncertainty of action values and on the distance of alternative cached action values. Overall, the model by default chooses on the basis of lighter model-free estimates, and integrates them with costly model-based predictions only when useful. Mental simulation uses a sampling method to produce reward expectancies, which are used to update the cached value of one or more actions; in turn, this updated value is used for the choice. The key predictions of the model are tested in different settings of a double T-maze scenario. Results are discussed in relation with neurobiological evidence on the hippocampus – ventral striatum circuit in rodents, which has been linked to goal-directed spatial navigation.

show abstract

Section: Resultssupporting

confidence: 74%

The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation

Pezzulo¹,

Rigoli²,

Chersi³

2013

Front. Psychol.

Self Cite

152

207

View full text Add to dashboard Cite

show abstract

“…Motivation also has been defined as the behaviorally-relevant processes that enable an organism to regulate its external and/or internal environments (Salamone, 1992(Salamone, , 2010Ryan and Deci, 2000). Motivational functions often involve sensory, motor, cognitive and emotional functions working together (Pezzulo and Castelfranchi, 2009;Salamone, 2010). Moreover, motivational processes are complex and multifaceted.…”

Section: Conceptual Background: Motivationmentioning

confidence: 99%

“…Moreover, motivational processes are complex and multifaceted. In general, motivationally relevant stimuli are at some physical or psychological distance from the organism (Ryan and Deci, 2000;Salamone, 2010;Salamone and Correa, 2012;Pezzulo and Castelfranchi, 2009). Thus, there are sequences of behaviors that bring the organism into physical proximity with the goal object (e.g., reward or reinforcer) or increase the chances that the goal object will be presented (appetitive, preparatory, approach, or "seeking" behavior), and then a terminal phase that involves the direct interaction with the motivational stimulus or goal object (i.e., consummatory or "taking" behavior; Craig, 1917;Markou et al, 2013).…”

Section: Conceptual Background: Motivationmentioning

confidence: 99%

Neurobiological basis of motivational deficits in psychopathology

Salamone

Koychev

Correa

et al. 2015

European Neuropsychopharmacology

View full text Add to dashboard Cite

“…Contrary to instrumental controllers, which learn novel actions guided by reward maximization (Daw et al, 2005; Pezzulo and Castelfranchi, 2009; Pezzulo and Rigoli, 2011; Solway and Botvinick, 2012), the Pavlovian system associates hard-wired reactions to unconditioned or conditioned stimuli (Mackintosh, 1983; Dayan and Seymour, 2008). The instrumental-Pavlovian interaction has been studied both in animals (Estes and Skinner, 1941; Rescorla and Solomon, 1967; Overmier et al, 1971; Dickinson and Pearce, 1977; Colwill and Rescorla, 1988; Holland, 2004) and, more recently, in humans (Bray et al, 2008; Talmi et al, 2008; Huys et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

Aversive Pavlovian Responses Affect Human Instrumental Motor Performance

Rigoli¹,

Pavone²,

Pezzulo³

2012

Front. Neurosci.

Self Cite

View full text Add to dashboard Cite

In neuroscience and psychology, an influential perspective distinguishes between two kinds of behavioral control: instrumental (habitual and goal-directed) and Pavlovian. Understanding the instrumental-Pavlovian interaction is fundamental for the comprehension of decision-making. Animal studies (as those using the negative auto-maintenance paradigm), have demonstrated that Pavlovian mechanisms can have maladaptive effects on instrumental performance. However, evidence for a similar effect in humans is scarce. In addition, the mechanisms modulating the impact of Pavlovian responses on instrumental performance are largely unknown, both in human and non-human animals. The present paper describes a behavioral experiment investigating the effects of Pavlovian conditioned responses on performance in humans, focusing on the aversive domain. Results showed that Pavlovian responses influenced human performance, and, similar to animal studies, could have maladaptive effects. In particular, Pavlovian responses either impaired or increased performance depending on modulator variables such as threat distance, task controllability, punishment history, amount of training, and explicit punishment expectancy. Overall, these findings help elucidating the computational mechanisms underlying the instrumental-Pavlovian interaction, which might be at the base of apparently irrational phenomena in economics, social behavior, and psychopathology.

show abstract

Intentional action: from anticipation to goal-directed behavior

Cited by 21 publications

References 29 publications

The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation

The Mixed Instrumental Controller: Using Value of Information to Combine Habitual Choice and Mental Simulation

Neurobiological basis of motivational deficits in psychopathology

Aversive Pavlovian Responses Affect Human Instrumental Motor Performance

Contact Info

Product

Resources

About