Feeder Approach between Trials Is Increased by Uncertainty and Affects Subsequent Choices

Gruber, Aaron J.; Thapa, Rajat; Randolph, Sienna H.

doi:10.1523/eneuro.0437-17.2017

Cited by 7 publications

(9 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…WSLS responding decreases with longer ITIs in rhesus monkeys (Deets, 1970 ) and in pigeons (Rayburn-Reeves et al, 2013 ). We did not find such a temporal relationship with win-stay in the present data, providing evidence supporting the hypothesis that win-stay and lose-shift are mediated by different neural mechanisms (Skelin et al, 2014 ; Gruber and Thapa, 2016 ; Gruber et al, 2017 ). We can only speculate on the information encoded by the decaying memory trace, but we suspect it is an inhibition of the reward position rather than an explicit representation of the reward omission.…”

Section: Discussionsupporting

confidence: 87%

“…We instead propose that lose-shift is mediated by sensorimotor systems, including the putamen in primates or LS in rodents. This is consistent with lesion data in rats (Skelin et al, 2014 ; Gruber et al, 2017 ), and impairments in a similar task (Rock-Paper-Scissors) in human patients with damage to the putamen (Danckert et al, 2012 ). Furthermore, it provides a parsimonious explanation for the increase in lose-shift in adults under load: the normal suppression of sensorimotor control by PFC (Jahanshahi et al, 1998 ; Knoch et al, 2005 ) is disrupted by the subtraction task, thereby unmasking lose-shift behavior mediated by sensorimotor system.…”

Section: Discussionsupporting

confidence: 87%

“…We have recently discovered (Skelin et al, 2014 ; Gruber et al, 2017 ) that lose-shift responding in rats depends on the lateral striatum (LS), a sensorimotor region of the striatum homologous to the putamen in primates (Johnston et al, 1990 ; Voorn et al, 2004 ; Balleine and O’Doherty, 2009 ). The PFC normally inhibits the behavioral control exerted by sensorimotor systems, which includes the expression of habits (Jahanshahi et al, 1998 , 2000 ; Knoch et al, 2005 ).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Lose-Shift Responding in Humans Is Promoted by Increased Cognitive Load

Ivan

Banks

Goodfellow

et al. 2018

Front. Integr. Neurosci.

Self Cite

View full text Add to dashboard Cite

The propensity of animals to shift choices immediately after unexpectedly poor reinforcement outcomes is a pervasive strategy across species and tasks. We report here on the memory supporting such lose-shift responding in humans, assessed using a binary choice task in which random responding is the optimal strategy. Participants exhibited little lose-shift responding when fully attending to the task, but this increased by 30%–40% in participants that performed with additional cognitive load that is known to tax executive systems. Lose-shift responding in the cognitively loaded adults persisted throughout the testing session, despite being a sub-optimal strategy, but was less likely as the time increased between reinforcement and the subsequent choice. Furthermore, children (5–9 years old) without load performed similarly to the cognitively loaded adults. This effect disappeared in older children aged 11–13 years old. These data provide evidence supporting our hypothesis that lose-shift responding is a default and reflexive strategy in the mammalian brain, likely mediated by a decaying memory trace, and is normally suppressed by executive systems. Reducing the efficacy of executive control by cognitive load (adults) or underdevelopment (children) increases its prevalence. It may therefore be an important component to consider when interpreting choice data, and may serve as an objective behavioral assay of executive function in humans that is easy to measure.

show abstract

Section: Discussionsupporting

confidence: 87%

Section: Discussionsupporting

confidence: 87%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Lose-Shift Responding in Humans Is Promoted by Increased Cognitive Load

Ivan

Banks

Goodfellow

et al. 2018

Front. Integr. Neurosci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although the win-stay and lose-shift are complementary response strategies, they are anatomically disassociated among goal-directed and sensorimotor systems. Lesions to the rodent lateral striatum (LS), which is homologous to the human putamen and essential for sensorimotor control (Parent and Hazrati, 1995 ), disrupt lose-shift responding but not win-stay (Skelin et al, 2014 ; Gruber et al, 2017 ; Thapa and Gruber, 2018 ). A similar shifting deficit has been observed in humans with damage to putamen or insula (Danckert et al, 2011 ).…”

Section: Introductionmentioning

confidence: 99%

“…A similar shifting deficit has been observed in humans with damage to putamen or insula (Danckert et al, 2011 ). Conversely, lesions of the rodent ventromedial striatum (VS), a key structure in goal-directed control that receives inputs from prefrontal cortex (Voorn et al, 2004 ), disrupts win-stay but not lose-shift responding (Gruber et al, 2017 ). Several other behavioral features in rodents and humans support this anatomical disassociation.…”

Section: Introductionmentioning

confidence: 99%

Cannabis use is associated with sexually dimorphic changes in executive control of visuospatial decision-making

Banks

Bennett

Sekuler

et al. 2022

Front. Integr. Neurosci.

Self Cite

View full text Add to dashboard Cite

When the outcome of a choice is less favorable than expected, humans and animals typically shift to an alternate choice option on subsequent trials. Several lines of evidence indicate that this “lose-shift” responding is an innate sensorimotor response strategy that is normally suppressed by executive function. Therefore, the lose-shift response provides a covert gauge of cognitive control over choice mechanisms. We report here that the spatial position, rather than visual features, of choice targets drives the lose-shift effect. Furthermore, the ability to inhibit lose-shift responding to gain reward is different among male and female habitual cannabis users. Increased self-reported cannabis use was concordant with suppressed response flexibility and an increased tendency to lose-shift in women, which reduced performance in a choice task in which random responding is the optimal strategy. On the other hand, increased cannabis use in men was concordant with reduced reliance on spatial cues during decision-making, and had no impact on the number of correct responses. These data (63,600 trials from 106 participants) provide strong evidence that spatial-motor processing is an important component of economic decision-making, and that its governance by executive systems is different in men and women who use cannabis frequently.

show abstract

Exploration-Exploitation and Suicidal Behavior in Borderline Personality Disorder and Depression

Tsypes,

Hallquist,

Ianni

et al. 2024

JAMA Psychiatry

View full text Add to dashboard Cite

ImportanceClinical theory and behavioral studies suggest that people experiencing suicidal crisis are often unable to find constructive solutions or incorporate useful information into their decisions, resulting in premature convergence on suicide and neglect of better alternatives. However, prior studies of suicidal behavior have not formally examined how individuals resolve the tradeoffs between exploiting familiar options and exploring potentially superior alternatives.ObjectiveTo investigate exploration and exploitation in suicidal behavior from the formal perspective of reinforcement learning.Design, Setting, and ParticipantsTwo case-control behavioral studies of exploration-exploitation of a large 1-dimensional continuous space and a 21-day prospective ambulatory study of suicidal ideation were conducted between April 2016 and March 2022. Participants were recruited from inpatient psychiatric units, outpatient clinics, and the community in Pittsburgh, Pennsylvania, and underwent laboratory and ambulatory assessments. Adults diagnosed with borderline personality disorder (BPD) and midlife and late-life major depressive disorder (MDD) were included, with each sample including demographically equated groups with a history of high-lethality suicide attempts, low-lethality suicide attempts, individuals with BPD or MDD but no suicide attempts, and control individuals without psychiatric disorders. The MDD sample also included a subgroup with serious suicidal ideation.Main Outcomes and MeasuresBehavioral (model-free and model-derived) indices of exploration and exploitation, suicide attempt lethality (Beck Lethality Scale), and prospectively assessed suicidal ideation.ResultsThe BPD group included 171 adults (mean [SD] age, 30.55 [9.13] years; 135 [79%] female). The MDD group included 143 adults (mean [SD] age, 62.03 [6.82] years; 81 [57%] female). Across the BPD (χ23 = 50.68; P &lt; .001) and MDD (χ24 = 36.34; P &lt; .001) samples, individuals with high-lethality suicide attempts discovered fewer options than other groups as they were unable to shift away from unrewarded options. In contrast, those with low-lethality attempts were prone to excessive behavioral shifts after rewarded and unrewarded actions. No differences were seen in strategic early exploration or in exploitation. Among 84 participants with BPD in the ambulatory study, 56 reported suicidal ideation. Underexploration also predicted incident suicidal ideation (χ21 = 30.16; P &lt; .001), validating the case-control results prospectively. The findings were robust to confounds, including medication exposure, affective state, and behavioral heterogeneity.Conclusions and RelevanceThe findings suggest that narrow exploration and inability to abandon inferior options are associated with serious suicidal behavior and chronic suicidal thoughts. By contrast, individuals in this study who engaged in low-lethality suicidal behavior displayed a low threshold for taking potentially disadvantageous actions.

show abstract

Feeder Approach between Trials Is Increased by Uncertainty and Affects Subsequent Choices

Cited by 7 publications

References 55 publications

Lose-Shift Responding in Humans Is Promoted by Increased Cognitive Load

Lose-Shift Responding in Humans Is Promoted by Increased Cognitive Load

Cannabis use is associated with sexually dimorphic changes in executive control of visuospatial decision-making

Exploration-Exploitation and Suicidal Behavior in Borderline Personality Disorder and Depression

Contact Info

Product

Resources

About