A mathematical model of reward and aversive nonreward: Its application in over 30 appetitive learning situations.

Daly, Helen B.; Daly, John T.

doi:10.1037/0096-3445.111.4.441

Cited by 99 publications

(117 citation statements)

References 127 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…In the model developed by Daly and Daly (1982), it is not unrealized expectation but a negative discrepancy between the strength of a conditioned approach tendency and the strength which the reinforcer will support that is assumed to generate a competing emotional The results of all three of these experiments yield readily to an equal-asymptote nonrepresentational analysis, which the results for vertebrates-however they may be understood-clearly do not. In Experiment 1, one of two targets contained 20 p.l of sucrose solution on Trials 1-16 and 5 p.lon Trials 17-32, while the opposite was true of the other target.…”

Section: Resultsmentioning

confidence: 82%

Learning in honeybees as a function of amount of reward: Tests of the equal-asymptote assumption

Buchanan

Bitterman

1989

Animal Learning & Behavior

View full text Add to dashboard Cite

In Experiments 1 and 2, honeybee foragers visiting the laboratory were fed on targets of two different colors, one containing 5 ILl and the other containing 20 ILl of 50% sucrose solution. The targets were presented singly in quasi-random sequences on the training visits, after which preference was measured in an unrewarded choice test. In Experiment 1, 16 differentially rewarded training trials with each color were followed by the same number of trials with the coloramount relation reversed; no preference for either color was found in the subsequent choice test. In Experiment 2, 20 differentially rewarded training trials with each color-enough to produce a clear preference for the 20-ILI color when given directly after pretraining-were given after 10 feedings to repletion on each color that were calculated to generate near-asymptotic associative strength; no preference for either color was found in the subsequent choice test. In Experiment 3, there were 12 feedings to repletion on one color and, on the other, 12 feedings to repletion followed by 15 trials with a small (5 ILl) reward; no preference was found in a subsequent choice test. The results of all three experiments support a nonrepresentational interpretation of the role of amount of reward in the learning of honeybees.In a continuing series of experiments on learning in honeybees as compared with learning in vertebrates (Bitterman, 1988), we began recently to study amount of reward (Buchanan & Bitterman, 1988), an important parameter oflearning in vertebrates (Mackintosh, 1974). Some previous work hadsuggested that learning in honeybees might be independent of amount of reward (Menzel & Erber, 1972), but the method employed in that work seemed inadequate for a variety of reasons which need not be considered again here. Our own method was to train honeybees with two targets of different colors that were presented successively in quasi-random sequences, one target always containing 5 ILl of a 50% sucrose solution, the other containing 20 ILl of the same solution, and then to measure persistence of response to the targets in an extinction test, that is, when they no longer contained sucrose. If the number of training trials with the two targets was the same, the animals responded more in extinction to the ZO-/Ll target than to the 5-ILI target, but the difference in amount of reward could be offset by a difference in frequency. If the number of 5-1L1 training trials was at least three times the number of 20-1L1 trials, the animals responded more in extinction to the 5-1L1 target; if the number of 5-1L1 training trials was twice the number of ZD-ILI trials, the animals responded equally often to the two targets in extinction.

show abstract

Section: Resultsmentioning

confidence: 82%

Learning in honeybees as a function of amount of reward: Tests of the equal-asymptote assumption

Buchanan

Bitterman

1989

Animal Learning & Behavior

View full text Add to dashboard Cite

show abstract

“…According to the representational view, frustration is generated by a discrepancy between the remembered consequences of response (in our experiments, sucrose solution of a given volume and concentration) and the new consequences (water). According to the nonrepresentational view, frustration is generated by a discrepancy between the associative strength developed in training with the old consequences and the associative strength that can be supported by the new ones (Daly & Daly, 1982). The present data do not seem to yield readily to either interpretation, pointing again to the need for a careful experimental analysis of the overlearning-extinction effect, after training both with single and with multiple stimuli, and in singlestimulus as well as in choice tests.…”

Section: Resultsmentioning

confidence: 39%

“…What will be required in any case is a proper moment-by-moment theory of extinction that will predict not only choice ratios, but absolute levels of responding. Always in the background of our thinking has been the possibility that the decline in resistance to extinction with increasing frequency of reward that has appeared in some of our experimentscan be understoodin terms of a modellike that developed by Daly and Daly (1982)to deal with an array of what seem to be frustration-based phenomena in rats, but the possibility may be more difficult to realize than we imagine. The Daly model itself has thus far been directed only at variousqualitative features of the rat data and has yet to be tested quantitatively.…”

Section: Resultsmentioning

confidence: 97%

Learning in honeybees as a function of amount and frequency of reward

Buchanan

Bitterman

1988

Animal Learning & Behavior

View full text Add to dashboard Cite

In a series of four experiments with free-flying honeybees, individual foragers were trained with targets of two different colors that contained 5 or 20 ILl of 50% sucrose solution. The two targets were singly presented in quasi-random sequences on each visit, with the amount of reward to be found on each target perfectly predictable from its color. The number of training visits (4-32) was varied both within and between experiments, and so also was the relative frequency of trials with the 5-and 20-ILI targets (1:1,2:1, 3:1, and 9:1). At the conclusion of training under each condition, unrewarded responses to the targets were measured in a 10-min extinction test, with the targets presented either separately to two different groups of animals (Experiment 1) or as a pair (Experiments 2-4). When the number of training trials with each target was the same (Experiments 1 and 2), the animals responded more in extinction to the 20-ILI target than to the 5-ILI target, although there was a decline in the overall level of responding to both targets (an overlearning-extinction effect) as the number of training trials increased. After nine times as many, or only three times as many, training trials with the 5'ILI target as with the 20-ILI target, the animals responded more in extinction to the 5-ILI target (Experiment 3); after twice as many training trials with the 5-ILI target as with the 20-ILI target, there was equal responding to both (Experiment 4). The preferences shown in the choice tests of Experiments 2-4 could be simulated rather accurately on the assumptions of a model previously developed to deal with the discrete-trials choice behavior of honeybees and the further assumption that associative strength grows at a rate increasing with amount of reward to an asymptote independent ofamount of reward.In a continuing series of experiments on learning in honeybees as compared with learning in vertebrates (Bitterman, 1988), we tum now to the role of amount of reward, a powerful determinant of performance in appetitive conditioning experiments with vertebrates (Mackintosh, 1974). About quality of reward (concentration of sucrose solution in unlimited amount), we already have some interesting information from work with free-flying honeybees (Couvillon & Bitterman, 1984; Shinoda & Bitterman, 1987): (1) After training with 50% sucrose on one target and 20% sucrose on another, the 50% target is preferred in a choice test; (2) resistance to extinction increases at first and then decreases as a function of the number of training visits to 50% sucrose (the overleaming-extinction effect), but there is no hint of nonmonotonicity when the training concentation is 20%; and (3) 20% sucrose is accepted less readily on a target that has always before contained 50% sucrose than on a target that has always before contained 20% sucrose (successive negative incentive contrast). These results are perfectly familiar, of course, from the study of vertebrate learning, although the underlying mechanisms may well be different. Successive negat...

show abstract

“…The model made some interesting predictions concerning when a preference for the unpredictable reward situation should occur. These predictions were subsequently supported (Daly, 1985;Daly & Daly, 1982); other theories have great difficulty accounting for these results (Daly, 1985(Daly, , 1986. We are just beginning to use the extensions of the model…”

mentioning

confidence: 84%

A computer simulation/mathematical model of learning: Extension of DMOD from appetitive to aversive situations

Daly

1987

Behavior Research Methods, Instruments, & Computers

Self Cite

View full text Add to dashboard Cite

The computer simulation/mathematical model called DMOD, which can simulate over 35 different phenomena in appetitive discrete-trial and simple free-operant situations, has been extended to include aversive di~rete-trial situations. Learning (V) is calculated using a three-parameter equation .1V = a{j(A -V) (see Daly & Daly, 1982;Rescorla & Wagner, 1972). The equation is applied to three possible goal events in the appetitive (e.g., food)case and to three in the aversive (e.g., shock) case. The original goal event can be present, absent, or reintroduced; in the appetitive situation, these events condition approach (Vap), avoidance (Vav), and courage (Vee), respectively. In the aversive situation, the events condition avoidance (Vav*), approach (Vap*), and cowardice (Vcc*), respectively. The model was developed in simple learning situations and subsequently was applied to complex situations. It can account for such diverse phenomena as contrast effects after reward shifts, greater persistence following partial than following continuous reinforcement, and a preference for predictable appetitive and predictable aversive events. Application of the aversive version of the model to "reward" shifts is described.Our goal is to develop a computer simulation/mathematical model of learning that is as simple as possible with as much breadth as possible. The model, called DMOD (Daly MODification of the Rescorla-Wagner Model), is simple because learning is calculated with one simple equation using three parameters. It is applied to diverse situations by assuming that there are a number of different goal events possible, each of which conditions either approach or avoidance of the goal. It was originally developed to account for appetitive, discrete-trial experiments, and can currently account for behavior in over 30 different paradigms (see Daly & Daly, 1982). DMOD was then extended to simulate the effect of simple schedules of reinforcement in free-operant experiments (Daly & Daly, 1984b). Our purpose is to outline the extension of DMOD to the aversive case and to show its application in a complicated experimental situation. To understand the rationale behind the extension, however, it is neces- sary to review development of the model in the appetitive case and the rules we follow for development of DMOD. SELECTION OF PHENOMENA TO BE SIMULATEDWe believe that the initial goal of a new model is to be able to account for well-established and replicable phenomena. We feel that it is dangerous to develop a model around recently discovered phenomena, because the boundary conditions under which they can be obtained and the variables influencing them are unknown. The purpose of a theory, however, is not only to integrate existing replicable data, but also to correctly predict new results. The primary phenomenon we use to test predictions of the appetitive version of DMOD is acquisition of a preference for predictable reward. The model made some interesting predictions concerning when a preference for the unpredictable reward situati...

show abstract

A mathematical model of reward and aversive nonreward: Its application in over 30 appetitive learning situations.

Cited by 99 publications

References 127 publications

Learning in honeybees as a function of amount of reward: Tests of the equal-asymptote assumption

Learning in honeybees as a function of amount of reward: Tests of the equal-asymptote assumption

Learning in honeybees as a function of amount and frequency of reward

A computer simulation/mathematical model of learning: Extension of DMOD from appetitive to aversive situations

Contact Info

Product

Resources

About