Reinforcement and Choice

Williams, Ben

doi:10.1016/b978-0-08-057169-0.50010-3

Cited by 34 publications

(28 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Instead it refers to the organism's matching of the relative frequency of choosing each alternative to the relative frequency of reinforcement). Thus, although there is some doubt about the validity of the matching law and its instantiation via the melioration mechanism as a general theory of human choice (see Savastano and Fantino, 1994;Tunney and Shanks, in press;Williams, 1994), it comfortably accounts for the behavior of those participants who maximize in the present experiments. On the other hand, it is not clear on this account why feedback and monetary payoff affect asymptotic levels of performance.…”

Section: Discussionmentioning

confidence: 86%

A re‐examination of probability matching and rational choice

Shanks

Tunney

McCarthy

2002

Behavioral Decision Making

241

244

View full text Add to dashboard Cite

In a typical probability learning task participants are presented with a repeated choice between two response alternatives, one of which has a higher payoff probability than the other. Rational choice theory requires that participants should eventually allocate all their responses to the high-payoff alternative, but previous research has found that people fail to maximize their payoffs. Instead, it is commonly observed that people match their response probabilities to the payoff probabilities. We report three experiments on this choice anomaly using a simple probability learning task in which participants were provided with (i) large financial incentives, (ii) meaningful and regular feedback, and (iii) extensive training. In each experiment large proportions of participants adopted the optimal response strategy and all three of the factors mentioned above contributed to this. The results are supportive of rational choice theory. Copyright # 2002 John Wiley & Sons, Ltd.key words probability matching; maximization; choice; rationality; feedback; payoffs; learning; reinforcement A striking violation of rational choice theory is commonly observed in simple repeated binary choice tasks in which a payoff is available with higher probability given one response than another. In such tasks people often tend to 'match' probabilities: That is, they allocate their responses to the two options in proportion to their relative payoff probabilities. Thus suppose that a monetary payoff of fixed size is given with probability p ¼ 0.7 for choosing left and with probability 1 À p ¼ 0.3 for choosing right. Probability matching refers to behavior in which left is chosen on about 70% of trials and right on 30%. Such responding violates rational choice theory because the optimal strategy in such tasks, after an initial period of experimentation and assuming that the payoff probabilities are stationary, is always to select the option associated with the higher probability of payoff. On any trial, the expected payoff for choosing left is higher than the expected payoff for choosing right.

show abstract

Section: Discussionmentioning

confidence: 86%

A re‐examination of probability matching and rational choice

Shanks

Tunney

McCarthy

2002

Behavioral Decision Making

241

244

View full text Add to dashboard Cite

show abstract

“…He argued that both models could be viewed as describing deviations from perfect matching as a function of temporal context. Because evidence suggests that matching may be the normative result in concurrent schedules, at least with exponentially distributed VI schedules (e.g., see Williams, 1988Williams, , 1994, it is tempting to speculate that matching may also be normative in concurrent chains when the terminal links are exponentially distributed VI schedules. If this is true, then Herrnstein's (1964) early argument, that matching could apply equally well to concurrent chains and concurrent schedules, would be valid.…”

Section: Discussionmentioning

confidence: 99%

The Matching Law and Amount‐dependent Exponential Discounting as Accounts of Self‐control Choice

Grace

1999

J Exper Analysis Behavior

View full text Add to dashboard Cite

Studies with humans have found evidence for amount-dependent temporal discounting, that is, that the sensitivity of choice to reinforcer delay varies inversely with reinforcer magnitude. To test whether similar results could be obtained with nonhumans, pigeons were trained on a two-component concurrent-chains procedure in which the durations of food reinforcement in the terminal links were equal within components but unequal between components. Terminal-link schedules were varied over four conditions to allow separate estimates of sensitivity to delay to be obtained for the large and small reinforcer-magnitude components. Although sensitivity to delay was greater in the smallmagnitude component for all subjects, the effect was quite small. The difference in sensitivity was generally less than one standard error, and t tests on parameter differences failed to reach significance. Several models for temporal discounting, including an amount-dependent exponential function, were fitted to the data from the first four conditions. The resulting parameter estimates were used to make predictions for a self-control condition in which one terminal link arranged a smaller, less delayed reinforcer and the other arranged a larger, more delayed reinforcer. For all models, predictions were considerably more accurate when sensitivity to delay was the same regardless of reinforcer magnitude. The results support the independence of delay and magnitude as required by a version of the matching law, and provide strong evidence against amount-dependent exponential discounting as an account of self-control choice. A new two-parameter discounting function, consistent with the matching law, is proposed that has wide empirical generality for both human and nonhuman data.

show abstract

“…Despite intensive research on a wide range of animal taxa, none of the major theories about the psychological processes that underlie matching satisfactorily explains all observed data, and there is still uncertainty as to whether matching or maximising is the basic choice rule (Williams 1994). However, evidence from both psychology and behavioural ecology indicates that fishes are capable of flexible exploitation strategies in response to reinforcement variation.…”

Section: Patch Use and Probability Matchingmentioning

confidence: 99%

Learning of Foraging Skills by Fish

Warburton

Hughes

2011

Fish Cognition and Behavior

View full text Add to dashboard Cite

IntroductionInvestigations of the role of learning and memory in the foraging behaviour of fishes are at an exciting stage. Although less thoroughly studied than traditional laboratory favourites such as rats, pigeons or bees, fish species are no longer to be consigned to the 'poorly understood' category. It is now possible to place the capacities of fish in the context of learning and memory as a whole, as evidenced by previous reviews such as those by Hart (1986Hart ( , 1993, Hughes et al. (1992) and Kieffer & Colgan (1992). The present chapter attempts to integrate perspectives and findings from the fields of behavioural ecology and comparative psychology. This approach has been adopted for the following reasons:1. Comparative psychology has revealed broad regularities in the general principles of learning across invertebrate and vertebrate taxa (Logue 1988; Domjan 1998) and across spatial and temporal domains (Cheng & Spetch 2001). The general principles that apply to learning in bees, pigeons and rats are likely to apply to fishes also. Psychology can clarify the mechanisms that underlie observed behaviour, while behavioural ecology can evaluate the adaptive significance of behavioural capacities demonstrated by psychology. 2. In several cases, static first-generation models based on optimal foraging theory (OFT) that do not represent temporal changes in internal state fail to predict observed behaviour (Hart 1993). More recent studies using more flexible (e.g. dynamic-programming) models have been more successful because of their ability to represent changes in internal state (e.g. physiology and learning), interactions between intrinsic and extrinsic variables and continuous behavioural adjustment in response to these factors (Ehlinger 1989;Kieffer & Colgan 1991;Hart 1993;Dall et al. 1999). Although standard OFT models predict that animals should exhibit all-or-nothing choice, experiments usually reveal partial preferences. It is likely that future models will draw increasingly on psychological effects (e.g. discrimination, memory, cue competition, interference and attention) to explain such divergences from the predictions of simple or idealised models and to test models based on risk and information (Shettleworth 1988).

show abstract

Reinforcement and Choice

Cited by 34 publications

References 61 publications

A re‐examination of probability matching and rational choice

A re‐examination of probability matching and rational choice

The Matching Law and Amount‐dependent Exponential Discounting as Accounts of Self‐control Choice

Learning of Foraging Skills by Fish

Contact Info

Product

Resources

About