Conditioned Reinforcement in Chain Schedules When Time to Reinforcement Is Held Constant

Abstract

Two alternative approaches describe determinants of responding to a stimulus temporally distant from primary reinforcement. One emphasizes the temporal relation of each stimulus to the primary reinforcer, with relative proximity of the stimulus determining response rate. A contrasting view emphasizes immediate consequences of responding to the stimulus, the key factor being the conditioned reinforcement value of those immediate consequences. To contrast these approaches, 4 pigeons were exposed to a two-component multiple schedule with three-link chain schedules in each component. Only middle-link stimuli differed between chains. Baseline reinforcement probabilities were 0.50 for both chains; during discrimination phases it was 1.0 for one chain and 0.0 for the other. During discrimination phases pigeons responded more to the reinforcement-correlated middle link than to the extinction-correlated middle link, demonstrating that responding was affected by the probability change. Terminal link responding was also higher in the reinforced chain, even though the terminal link stimulus was identical in both chains. Of greatest interest is initial link responding, which was temporally most distant from reinforcement. Initial link responding, necessarily equal in the two chains, was significantly higher during the 1.0/0.0 discrimination phases, even though overall reinforcement probability remained constant. For 3 of 4 birds, in fact, initial-link response rates were higher than terminal-link response rates, an outcome that can be ascribed only to the potent conditioned reinforcement properties of the middle-link stimulus during the discrimination phases. Results are incompatible with any account of chain behavior based solely on relative time to reinforcement.

When good news leads to bad choices

“…Suboptimal choice as evidenced in the procedures summarized above provides a challenge for models of choice that minimize or eliminate the role of conditioned reinforcement (Davison & Baum, ; Shahan, , ; see Bell & McDevitt, ). Other lines of evidence make similar arguments for the concept of conditioned reinforcement as incorporated in the SiGN hypothesis (e.g., Bell & Williams, ; Williams, ).…”

A theory of behavioral contrast

“…My hypothesis is that, with a VI schedule, some instances of reinforcement will occur soon after component transition, and they will do more than their share in strengthening alternate responses in the focal component (Killeen, ). Furthermore, the signaled extinction condition is not very different from the three‐link chain schedule studied by Bell and Williams () in which the middle link, when predictive of a delayed reinforcer at the end of the third link, was demonstrated to have “potent conditioned reinforcement properties” (p. 179). Thus, although the pigeons learned that they need not respond during the EXT/delay stimulus, that stimulus signaled the same distribution of delays as did the VI stimulus, and its onset may have been equally reinforcing or punishing as that stimulus.…”

“…One could suppose that his contrast effect was primarily initial contrast—but his experimental conditions ruled this out. One could suppose that the schedule transitions were not conditioned reinforcers or punishers—but Williams's own work makes this unlikely (Bell & Williams, ; Williams, ). In any case, if the schedule transitions were not functional conditioned reinforcers or punishers, that alone would undermine the machinery necessary for the present theory.…”

Go forth and be variable

“…Instead, these consequent stimuli provide ''signpost'' information about, for example, the time to the next occurrence of a phylogenetically important event (e.g., the delivery of food). Other researchers are skeptical of this discriminative-control position and conduct important research that explores the assumptions of old and new accounts of this process (e.g., Bell & Williams, 2013). The debate continues and, in my opinion, it is not an esoteric debate; if we better understood the ''reinforcement'' process, the translational research that followed would, I presume, be important indeed.…”