Pigeons show near-optimal win-stay/lose-shift performance on a simultaneous-discrimination, midsession reversal task with short intertrial intervals

Abstract

Discrimination reversal tasks have been used as a measure of species flexibility in dealing with changes in reinforcement contingency. The simultaneous-discrimination, midsession reversal task is one in which one stimulus (S1) is correct for the first 40 trials of an 80-trial session and the other stimulus (S2) is correct for the remaining trials. After many sessions of training with this task, pigeons show a curious pattern of choices. They begin to respond to S2 well before the reversal point (they make anticipatory errors) and they continue to respond to S1 well after the reversal (they make perseverative errors). That is, they appear to be using the passage of time or number of trials into the session as a cue to reverse. We tested the hypothesis that these errors resulted in part from a memory deficit (the inability to remember over the intertrial interval, ITI, both the choice on the preceding trial and the outcome of that choice) by manipulating the duration of the ITI (1.5, 5, and 10 s). We found support for the hypothesis as pigeons with a short 1.5-s ITI showed close to optimal win-stay/lose-shift accuracy.

The Organization of Behavior over Time: Insights from Mid-session Reversal

“…The latter result suggests the birds were likely using temporal information from across a number of sessions to compute an aggregate expectation of when Task 1 or 2 would be in effect. Similar molar aggregations from across sessions seem to have occurred in other studies using variable reversal locations as well (McMillan et al, 2014; Rayburn-Reeves, Laude, et al, 2013). In fact, any anticipation of a reversal within a session, variable or not, represents this type of molar aggregation operation from across prior sessions.…”

“…Many studies have attempted to reduce the relevancy and predictability of the timing cue by randomly varying across sessions the trial at which the discrimination reversal occurs (McMillan, Kirk, & Roberts, 2014; McMillan, Sturdy, Pisklak, & Spetch, 2016; Rayburn-Reeves, Laude, & Zentall, 2013; Rayburn-Reeves et al, 2011; Rayburn-Reeves & Zentall, 2013; A. P. Smith, Pattison, & Zentall, 2016).…”

“…In an attempt to better clarify the role of the memory for prior trials versus sustained postural or location orientation during the discriminations, Rayburn-Reeves, Laude, et al (2013) manipulated the time between trials to see whether shorter or longer ITI durations would produce more efficient use of the previous trials’ outcomes. One hypothesis based on a theory of memory decay is that if the animal is given too much time between trials it may not be able to remember its most recent experiences because of the decay in memory that occurs over time.…”

Behavioral flexibility is increased by optogenetic inhibition of neurons in the nucleus accumbens shell during specific time segments

“…Such a framework combines rule-based win-stay, lose-shift models (Worthy et al 2012) with reinforcement learning models (Sutton and Barto 1981;Barto and Sutton 1982) by basing decisions on the recency-weighted average reward, leading to selection of the option with greatest expected reward values (Worthy et al 2013). Optimal behavior in such contingency switch tasks is possible using a win-stay, loseshift strategy that repeats the response from the last trial if it was correct but switches to the alternative response if it was incorrect (Rayburn-Reeves et al 2013). Such a strategy requires integration of the results of previous responses into subsequent responses.…”

“…Tasks that include a switch in contingencies from a previously reinforced response to another response provide a sensitive measure of behavioral flexibility (Bitterman 1975;Kehagia et al 2010;Rayburn-Reeves et al 2013). The nucleus accumbens (NAcc) plays an important role in several forms of behavioral flexibility, including latent inhibition, attentional set shifting, and reversal learning (Stern and Passingham 1995;Cools et al 2006;Floresco et al 2006;O'Neill and Brown 2007), with the core and shell subregions of the NAcc regulating separate components (Weiner et al 1996;Parkinson et al 1999;Corbit et al 2001;Ito et al 2004;Cardinal and Cheung 2005;Pothuizen et al 2005a,c;Granon and Floresco 2009).…”