The Memory Trace Supporting Lose-Shift Responding Decays Rapidly after Reward Omission and Is Distinct from Other Learning Mechanisms in Rats

Abstract

The propensity of animals to shift choices immediately after unexpectedly poor reinforcement outcomes is a pervasive strategy across species and tasks. We report here that the memory supporting such lose-shift responding in rats rapidly decays during the intertrial interval and persists throughout training and testing on a binary choice task, despite being a suboptimal strategy. Lose-shift responding is not positively correlated with the prevalence and temporal dependence of win-stay responding, and it is inconsistent with predictions of reinforcement learning on the task. These data provide further evidence that win-stay and lose-shift are mediated by dissociated neural mechanisms and indicate that lose-shift responding presents a potential confound for the study of choice in the many operant choice tasks with short intertrial intervals. We propose that this immediate lose-shift responding is an intrinsic feature of the brain’s choice mechanisms that is engaged as a choice reflex and works in parallel with reinforcement learning and other control mechanisms to guide action selection.

Lose-Shift Responding in Humans Is Promoted by Increased Cognitive Load

“…WSLS responding decreases with longer ITIs in rhesus monkeys (Deets, 1970 ) and in pigeons (Rayburn-Reeves et al, 2013 ). We did not find such a temporal relationship with win-stay in the present data, providing evidence supporting the hypothesis that win-stay and lose-shift are mediated by different neural mechanisms (Skelin et al, 2014 ; Gruber and Thapa, 2016 ; Gruber et al, 2017 ). We can only speculate on the information encoded by the decaying memory trace, but we suspect it is an inhibition of the reward position rather than an explicit representation of the reward omission.…”

“…The probability of lose-shift responding decayed with increasing delays between the feedback signal and the next choice. This decay is very similar to that of the lose-shift observed in rodents performing against the same computer algorithm (Gruber and Thapa, 2016 ). The time dependence is consistent with findings in other species.…”

“…If lose-shift is mediated by this conserved structure, we expect it to present in many species. We propose that the reason rats lose-shift more on the analogous task (68 ± 1%; Gruber and Thapa, 2016 ) is that they have a more primitive PFC that does not hold the same level of cognitive control over sensorimotor systems, leaving them to rely more on this reflex responding.…”

“…We next sought to determine if the memory trace supporting lose-shift responses decays with time, as has been reported in rats (Gruber and Thapa, 2016 ). We found that the lose-shift probability among the group with the cognitive load does indeed decay with increasing (ITI; F vs. constant model = 13.4, df = 4, p = 0.02; r 2 = 0.71; Figure 1D ).…”

Variability in competitive decision-making speed and quality against exploiting and exploitative opponents

“…In particular, lose-shift appears to be a more robust phenomenon than win-stay, possibly in part due to the high cost of ‘losing’ from an evolutionary perspective 34 , 35 . Fundamental differences between reinforcement learning principles are further supported by animal work in which lose-shift mechanisms are also anatomically distinct from win-stay mechanisms (c.f., lesioning of the ventrolateral striatum), where lose-shift represents a “choice reflex” within the animal brain 36 , p. 1 37 . Differences in the degree to which win-stay and lose-shift behaviour are under cognitive control are similarly reflected in human work in which responses following wins tend to approach MS whereas responses following losses reveal a higher-than-expected level of shift behavior 3 , 6 .…”