Algorithmic Shaping and Misbehavior in the Acquisition of Token Deposit by Rats

Abstract

In two experiments, rats were trained to deposit ball bearings down a hole in the floor, using an algorithmic version of shaping. The experimenter coded responses expected to be precursors of the target response, ball bearing deposit; a computer program reinforced these responses, or not, according to an algorithm that mimicked the processes thought to occur in conventional shaping. In the first experiment, 8 of 10 rats were successfully shaped; in the second, 5 of 5 were successfully shaped, and the median number of sessions required was the same as for a control group trained using conventional shaping. In both experiments, "misbehavior," that is, excessive handling and chewing of the ball bearings, was observed, and when the algorithmic shaping procedure was used, misbehavior could be shown to occur in spite of reduced reinforcement for the responses involved.

Maintenance of responding when reinforcement becomes delayed

“…In contrast, Group HH might have suffered from a vicious circle effect: The less they responded, the lower the reinforcement rate and, as a consequence, the lower their activity level. This is a challenge when shaping new behavior (see, e.g., Midgley, Lea, & Kirby, 1989;Pear & Legris, 1987). Such a vicious circle is not a problem when interpreting Harker's (1956) results, since his procedure ensured that all of his rats received a food pellet on every trial.…”

Learned variability

“…Response rate (Alleman & Zeiler, 1974;Bickel, Higgins, Kirby, & Johnson, 1988;Weiner, 1964Weiner, , 1969, persistence (Eisenberger, 1992;Seligman & Maier, 1967) and topography (Midgley, Lea, & Kirby, 1989;Muenzinger et al, 1929;Stokes & Balsam, 1991) are all influenced by initial conditions. For example, Stokes and Balsam showed that preselected approximations reinforced early in shaping remain in rats' repertoires even when they are no longer required by the reinforcement contingency.…”

Beyond the “hot-and-cold” game: A demonstration of computer-controlled shaping

“…In the present demonstration, a single set ofparameters was used throughout the session. Thus, besides serving as a classroom demonstration that allows students to explore the science underlying the art ofshaping (Midgley et al, 1989;Pear & Legris, 1987), the present program may allow us to examine questions important to our understanding of the shaping process itself.…”

“…Second, the person saying "hot" and "cold" often dispenses these reinforcers and punishers in a series of gradations by saying things like "hot," "hotter," "very hot," or "cold," "colder," "very cold." Although there is nothing in the definition of shaping that prohibits using different magnitudes of reinforcement, doing so in the hot-and-cold game makes it more difficult to relate the game to empirical studies (e.g., Eckerman, Hienz, Stern, & Kowlowitz, 1980;Midgley et al, 1989;Pear & Legris, 1987;Platt, 1973) and, to a lesser extent, conceptual analyses of shaping (e.g., Catania, 1998;Donahoe et al, 1993;Stokes, 1995). As it has been used in countless studies and illustrated in as many textbooks, the intensity of the reinforcer is held constant during shaping.…”