Signal properties of reinforcement and reinforcement omission on a multiple fixed-ratio schedule

Abstract

A series of experiments used food-deprived pigeons to examine several parameters of reinforcement omission in an attempt to control changes of keypeck response measures on a subsequent schedule, In Experiments 1 and 2, the pigeons were tested with a multiple fixed-ratio schedule on which reinforcement was occasionally omitted at the completion of the first component, The duration of the delay occurring in lieu of reinforcement was systematically varied. In Experiment 3, the stimulus that signaled the second component of the schedule was altered to appear either more or less similar to the stimulus that signaled the first component. Two principal results are reported: (1) Response latency decreased and, to a much lesser extent, terminal response rate increased as the delay occurring in lieu of reinforcement decreased; and (2) both latency decrease and response-rate increase were enhanced by a second component stimulus which was similar to the first. The results are evaluated in terms of Amsel's frustration theory and an analysis by Staddon which suggests that reinforcement inhibits responding. The data appear to support Staddon's argument that rate increases and latency decreases following reinforcement omission are largely a function of an attenuation of the inhibitory influence of reinforcement, an effect that is enhanced by stimulus generalization. Accordingly, it is proposed that an animal's response to reinforcement omission is determined by a stimulus complex that minimally includes the omission event and component cues.A number of studies have demonstrated an increase in response rate on the second "half" of a trial following the omission of an appetitive reinforcer on the first half. This effect has been reported for rats in a double alley (Amsel, 1958;Matzel, 1984), pigeons on a multiple fixedinterval (PI) schedule (Staddon & Innis, 1966, 1969, and monkeys on a multiple fixed-ratio (FR) schedule (Davenport & Thompson, 1965). Amsel (1962) has attributed this pattern of results to an active, motivational response evoked by the omission of an expected reinforcer. Staddon (1974), however, has argued that this effect is largely due to the attenuation of the inhibitory influence of reinforcement that occurs on an omission trial.At least in those operant procedures that utilize FI schedules, available evidence suggests that the apparent increase in response rate following occasional reinforcement omission is the result of the removal of the inhibitory properties of reinforcement rather than enhanced motivation. Fixed-interval schedules typically produce a pause followed by gradually increased responding as the interval progresses, a pattern commonly referred to as a tern, response latency and the ensuing response gradient are greatly diminished following instances of reinforcement omission relative to instances of reinforcement (Staddon & Innis, 1969). The net result is an apparently higher response rate than is seen following reinforcement, an increase which Staddon and Innis have argued is due to the attenuat...

A Survey of Scalable Reinforcement Learning

“…The simple local rules play a significant part in generating emergent flocking alongside other behaviors [22]. Among the mentioned local methods, there has been an active exploration of the social forces model [23]. The pedestrian dynamics point of view illustrates that the various emergent behaviors have been effectively analyzed in some of the most popular microscopic models, such as the social forces, cellular automata, as well as agent-based models [24].…”