Learning by pigeons playing against tit-for-tat in an operant prisoner’s dilemma

Abstract

Each of four pigeons was exposed to a single random-ratio schedule of reinforcement in which the probability of reinforcement for a peck on either of two keys was 1/25. Reinforcer amounts were determined by an iterated prisoner's dilemma (IPD) matrix in which the "other player" (a computer) played tit-for-tat. One key served as the cooperation (C) key; the other served as the defection (D) key. If a peck was scheduled to be reinforced and the D-key was pecked, the immediate reinforcer of that peck was always higher than it would have been had the C-key been pecked. However, if the C-key was pecked and the following peck was scheduled to be reinforced, reinforcement amount for pecks on either key were higher than they would have been if the previous peck had been on the D-key. Although immediate reinforcement was always higher for D-pecks, the overall reinforcement rate increased linearly with the proportion of C-pecks. C-pecks thus constituted a form of self-control. All the pigeons initially defected with this procedure. However, when feedback signals were introduced that indicated which key had last been pecked, cooperation (relative rate of C-pecks)-hence, self-control-increased for all the pigeons.

Short-term gains, long-term pains: How cues about state aid learning in dynamic environments

“…For example, Herrnstein et al (1993, Experiment 1) used a task similar to our own and found that participants were more likely to maximize when provided with an arrow that indicated the number of responses the participant made to the maximizing choice option over the relevant choice history. Similarly, Sanabria et al (2003) had pigeons “play” the iterated prisoner’s dilemma against a computer opponent that followed a tit-for-tat strategy (Axelrod, 1984). Consistent with work in the maximization/melioration paradigm, pigeons preferred a short-term “defect” strategy on every trial.…”

Temporal Dynamics of Cooperation

“…Baker and Rachlin () found that pigeons could learn to consistently cooperate versus tit for tat when delays were bridged over with discriminative stimuli that signaled to the pigeons their own prior choice (also see Sanabria, Baker, & Rachlin, ). Stephens, McLinn, and Stevens () studied bluejays playing a prisoner's dilemma game versus tit for tat.…”

The Importance of Being Active

“…Such spontaneous activity is both a necessary prerequisite and an inevitable consequence of evolution. Ultimately, the conceptually simple process of generating activity and evaluating its consequences forms one of the fundamental cornerstones not only for all of our human nature, but also for our social coherence: human nature as described in planning, willing, and controlling our behavior (Frith et al, 1991;Knight et al, 1995;Lezak, 1995;Owen, 1997;Wegner, 2002) and our social coherence, as based on cooperation (Gutnisky and Zanutto, 2004a;McNamara et al, 2004;Sanabria et al, 2003). Martin Heisenberg is a visionary and a pioneer in the neurobiological study of these and related neural processes.…”