Instrumental Divergence and the Value of Control

Mistry, Prachi; Liljeholm, Mimi

doi:10.1038/srep36295

Cited by 11 publications

(16 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These findings suggest that dopamine signals targeting areas in the striatum may be important for reinforcing the preference for control. Additionally, a recent study demonstrated that a preference for free choice was more pronounced in case of high instrumental divergence (i.e., when choices differ more with respect to their outcome probability distributions) (Mistry and Liljeholm, 2016). Thus, the preference for free choice was higher when choices have a more meaningful impact on outcome.…”

Section: Instrumental Contingency Shapes Controllabilitymentioning

confidence: 99%

A Reward-Based Framework of Perceived Control

Wang

Bhanji

et al. 2019

Front. Neurosci.

View full text Add to dashboard Cite

Perceived control can be broadly defined as the belief in one’s ability to exert control over situations or events. It has long been known that perceived control is a major contributor toward mental and physical health as well as a strong predictor of achievements in life. However, one issue that limits a mechanistic understanding of perceived control is the heterogeneity of how the term is defined in models in psychology and neuroscience, and used in experimental settings across a wide spectrum of studies. Here, we propose a framework for studying perceived control by integrating the ideas from traditionally separate work on perceived control. Specifically, we discuss key properties of perceived control from a reward-based framework, including choice opportunity, instrumental contingency, and success/reward rate. We argue that these separate reward-related processes are integral to fostering an enhanced perception of control and influencing an individual’s behavior and well-being. We draw on select studies to elucidate how these reward-related elements are implicated separately and collectively in the investigation of perceived control. We highlight the role of dopamine within corticostriatal pathways shared by reward-related processes and perceived control. Finally, through the lens of this reward-based framework of perceived control, we consider the implications of perceived control in clinical deficits and how these insights could help us better understand psychopathology and treatment options.

show abstract

Section: Instrumental Contingency Shapes Controllabilitymentioning

confidence: 99%

A Reward-Based Framework of Perceived Control

Wang

Bhanji

et al. 2019

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Experimental design and statistical analysis Task and procedure. Participants were scanned with fMRI while performing a simple gambling task, illustrated in Figure 1 and described in detail by Mistry and Liljeholm (2016). At the start of the experiment, participants were instructed that they would assume the role of a gambler in a casino, playing a set of four slot machines (labeled A1, A2, A3, and A4, respectively) that yielded three different colored tokens (blue, green, and red), each worth a particular amount of money, with different probabilities.…”

Section: Methodsmentioning

confidence: 99%

“…A series of recent studies (Mistry and Liljeholm, 2016;Liljeholm et al, 2018) have demonstrated that individuals prefer environments in which instrumental divergence, the degree to which alternative actions differ with respect to their outcome probability distributions, is relatively high. A high level of instrumental divergence is a necessary feature of flexible instrumental control: If all available action alternatives have identical, or very similar, outcome distributions, such that selecting one action over another does not significantly alter the probability of any given outcome state, an agent's ability to exert control over its environment is considerably impaired.…”

Section: Introductionmentioning

confidence: 99%

The Rostrolateral Prefrontal Cortex Mediates a Preference for High-Agency Environments

Norton

Liljeholm

2020

J. Neurosci.

Self Cite

View full text Add to dashboard Cite

The ability to exert flexible instrumental control over one's environment is a defining feature of adaptive decision-making. Here, we investigated neural substrates mediating a preference for environments with greater instrumental divergence, the distance between outcome probability distributions associated with alternative actions. A formal index of agency, instrumental divergence allows an organism to flexibly obtain the currently most desired outcome as preferences change. As such, it may have intrinsic utility, guiding decisions toward environments that maximize instrumental power. Consistent with this notion, we found that a measure of expected value that treats instrumental divergence as a reward surrogate provided a better account of male and female human participants' choice preferences than did a conventional model, sensitive only to monetary reward. Using model-based fMRI, we found that activity in the rostrolateral and ventromedial PFC, regions associated with abstract cognitive inferences and subjective value computations, respectively, scaled with the divergence-based account of expected value. Implications for a neural common currency of information theoretic and motivational variables are discussed.

show abstract

“…In the counterfactual literature, models of causal reasoning (e.g., Pearl, 2000) share this idea with modern instantiations of the associative approach, such as recent accounts based on experienced action-outcome contingency -where contingency is defined as the difference between conditional probabilities, such as the so-called "ΔP rule" (e.g., Tanaka, Balleine and O'Doherty, 2008) -and with model-based learning algorithms drawing upon the notion of instrumental divergence (i.e., "Jensen-Shannon divergence"). Instrumental divergence formalizes the causal power of an action as the difference between probabilities of a given outcome in the presence vs. absence of this action (Liljeholm et al, 2011;Liljeholm et al, 2013;Mistry and Liljeholm, 2016). Interestingly, both counterfactual reasoning and instrumental divergence are endowed with the same prior belief about goal-directed actions.…”

Section: Counterfactual Models: Causation Is About Actions That Make mentioning

confidence: 99%

“…CF studies have provided convincing evidence that people generate counterfactuals when reasoning about causation (Sloman and Lagnado, 2015, for a review), whereas instrumental divergence provides a learning rule for how people make choices based on maximized divergence (Mistry & Liljeholm, 2016). However, both views have shortcomings.…”

Section: Overview Of the Present Studymentioning

confidence: 99%

Believing in one's power: a counterfactual heuristic for goal-directed control

Chambon

Théro

Findling

et al. 2018

Preprint

View full text Add to dashboard Cite

Most people envision themselves as operant agents endowed with the capacity to bring about changes in the outside world. This ability to monitor one's own causal power has long been suggested to rest upon a specific model of causal inference, i.e., a model of how our actions causally relate to their consequences. What this model is and how it may explain departures from optimal inference, e.g., illusory control and self-attribution biases, are still conjecture. To address this question, we designed a series of novel experiments requiring participants to continuously monitor their causal influence over the task environment by discriminating changes that were caused by their own actions from changes that were not. Comparing different models of choice, we found that participants' behaviour was best explained by a model deriving the consequences of the forgone action from the current action that was taken and assuming relative divergence between both. Importantly, this model agrees with the intuitive way of construing causal power as "difference-making" in which causally efficacious actions are actions that make a difference to the world. We suggest that our model outperformed all competitors because it closely mirrors people's belief in their causal power -a belief that is well-suited to learning action-outcome associations in controllable environments. We speculate that this belief may be part of the reason why reflecting upon one's own causal power fundamentally differs from reasoning about external causes.1 The need to be and feel in control is so strong that individuals do whatever they can to re-establish control when it disappears or is taken away (Brehm, 1966;Brehm and Brehm, 1981). Reestablishment of lost agency can take different forms from illusory pattern perception to erroneous identification of a causal relationship between random or unrelated stimuli. Thus, people experiencing a loss of control are more likely to see images in noise, to form illusory correlations, to perceive conspiracies or to develop superstitions (Whitson & Galinsky, 2008). Such erroneous causal attributions would help restore 4 Here we draw upon a classical distinction between associative and generative approaches to causation, according to which causes are "associated" with effects by retrospection or actively "generate" their effects through an operant mechanism (e.g., Cheng, 1997). Strictly speaking, however, associative models in the form of reinforcement-learning (RL) algorithms do also possess a generative model of the world (i.e., an explanation for how observations are generated), whereas counterfactual emulation is a generative mechanism per se (i.e., a mechanism to decide which among several candidate causes has generated the effect). Here, and in what follows, we take the "generative" term in a broader and more liberal sense: generative models are those models drawing on an explicit representation of the generative source (usually in the form of a probability distribution over action outcomes), which can be used to make ...

show abstract

Instrumental Divergence and the Value of Control

Cited by 11 publications

References 56 publications

A Reward-Based Framework of Perceived Control

A Reward-Based Framework of Perceived Control

The Rostrolateral Prefrontal Cortex Mediates a Preference for High-Agency Environments

Believing in one's power: a counterfactual heuristic for goal-directed control

Contact Info

Product

Resources

About