The paradoxical effect of low reward probabilities in suboptimal choice.

Abstract

When offered a choice between 2 alternatives, animals sometimes prefer the option yielding less food. For instance, pigeons and starlings prefer an option that on 20% of the trials presents a stimulus always followed by food, and on the remaining 80% of the trials presents a stimulus never followed by food (the Informative Option), over an option that provides food on 50% of the trials regardless of the stimulus presented (the Noninformative Option). To explain this suboptimal behavior, it has been hypothesized that animals ignore (or do not engage with) the stimulus that is never followed by food in the Informative Option. To assess when pigeons attend to the stimulus usually not followed by food, we increased the probability of reinforcement, p, in the presence of that stimulus. Across 2 experiments, we found that the value of the Informative Option decreased with p. To account for the results, we added to the Reinforcement Rate Model (and also to the Hyperbolic Discounting Model) an engagement function, f(p), that specified the likelihood the animal attends to a stimulus followed by reward with probability p, and then derived the model predictions for 2 forms of f(p), a linear function, and an all-or-none threshold function. Both models predicted the observed findings with a linear engagement function: The higher the probability of reinforcement after a stimulus, the higher the probability of engaging the stimulus, and, surprisingly, the less the value of the option comprising the stimulus. (PsycINFO Database Record

The Δ–∑ hypothesis: How contrast and reinforcement rate combine to generate suboptimal choice

“…Increasing p 1,2 retarded acquisition, but it did not change steady‐state preference for the suboptimal alternative. This result challenges most models mentioned above (but see the latest version of RRM; Fortes et al, ) because they do not explain how each terminal link is combined to determine the alternative's value. For instance, it is not clear how to combine the value of a signal with probability p 1,1 = 1 and a signal of probability p 1,2 ≠ 0 to determine the subjective value of alternative 1 (e.g.…”

“…Hence, the overall probability of receiving food given a peck at the Circle key equals 0.5. Pigeons and starlings consistently prefer the Plus (suboptimal) alternative to the Circle (optimal) alternative (Fortes et al, ; Fortes et al, ; Fortes et al, ; Stagner et al, ; Stagner & Zentall, ; Vasconcelos et al, ; Zentall & Stagner, ).…”

“…Fortes et al () increased p 1,2 from 0 to .375 while maintaining constant the remaining probabilities (p 1,1 = 1,p 2,1 = p 2,2 = .5). Increasing p 1,2 retarded acquisition, but it did not change steady‐state preference for the suboptimal alternative.…”

Suboptimal choice and initial‐link requirement

“…While there has been a recent wave of articles published on suboptimal choice (for reviews, see McDevitt et al, 2016;Vasconcelos et al, 2018;and Zentall, 2016), most of the focus has been placed on the role of the terminal links in determining preference (e.g., Fortes, Pinto, Machado, & Vasconcelos, 2018;Fortes, Vasconcelos, & Machado, 2016;McDevitt, Pisklak, Spetch, & Dunn, 2018;Pisklak et al, 2015;Stagner et al, 2012;Stagner & Zentall, 2010). In most studies of suboptimal choice the initial-link schedule is an FR 1.…”

Frequency and value both matter in the suboptimal choice procedure

“…Another recent model, based on considerations of foraging behavior in the natural environment, similarly predicts that reinforcement frequency on a signaled alternative should have no effect on preference (Fortes, Machado, & Vasconcelos, 2017;Fortes, Pinto, Machado, & Vasconcelos, 2018;Fortes, Vasconcelos, & Machado, 2016;Vasconcelos, Machado, & Pandeirada, 2018). One formalization of this functional model, called the reinforcement rate model (RRM) focuses on the rate of energy intake in the presence of informative (i.e., signaled) stimuli and noninformative (unsignaled) stimuli (Fortes et al, 2017).…”