The attentional learning theory of Pearce and Hall (1980) predicts more attention to uncertain cues that have caused a high prediction error in the past. We examined how the cue-elicited pupil dilation during associative learning was linked to such errordriven attentional processes. In three experiments, participants were trained to acquire associations between different cues and their appetitive (Experiment 1), motor (Experiment 2), or aversive (Experiment 3) outcomes. All experiments were designed to examine differences in the processing of continuously reinforced cues (consistently followed by the outcome) versus partially reinforced, uncertain cues (randomly followed by the outcome). We measured the pupil dilation elicited by the cues in anticipation of the outcome and analyzed how this conditioned pupil response changed over the course of learning. In all experiments, changes in pupil size complied with the same basic pattern: During early learning, consistently reinforced cues elicited greater pupil dilation than uncertain, randomly reinforced cues, but this effect gradually reversed to yield a greater pupil dilation for uncertain cues toward the end of learning. The pattern of data accords with the changes in prediction error and error-driven attention formalized by the Pearce-Hall theory. K E Y W O R D Sassociative learning, attention, human fear conditioning, pupil dilation, reward | IN TRO DUCT IO NAssociative learning theories assume that repeated observations of the co-occurrence of two external events will lead to the formation of an association that links the representation of both events in memory (Pearce & Bouton, 2001;Rescorla, 1988). However, the association between cues and a significant outcome like reward (or punishment) often is acquired in an uncertain environment, where the contingency between cues and the outcome is not perfect but rather probabilistic. From this perspective, the learning process involves two important variables: (a) the expectancy of the outcome given the cue, and (b) the uncertainty of that expectation. For example, if three different cues were followed by reward on 0%, 50%, and 100% of their past occurrences, these cues differ with respect to reward expectancy and uncertainty. The 0% cue elicits low reward expectancy and low uncertainty. The 100% cue elicits high reward expectancy and low uncertainty. The 50% cue is partially reinforced and elicits an intermediate reward expectancy but high uncertainty whether reward will occur or not. From the perspective of associative learning theories, this uncertainty results from the fact that the 50% cue in the past had been associated with a higher prediction error, whereas in contrast this error in predicting the omission or occurrence of the outcome was small for the 0% cue and the 100% cue.

In two predictive-learning experiments, we investigated the role of the informational value of contexts for the formation of context-dependent behavior. During Phase 1 of each experiment, participants received either a conditional discrimination in which contexts were relevant (Group Relevant) or a simple discrimination in which contexts were irrelevant (Group Irrelevant). Each experiment also included an ABA renewal procedure. Participants received Z+ in context A during Phase 1, extinction of Z in context B during Phase 2, and were tested with Z in context A during a test phase. In each experiment, extinction of Z proceeded faster and was followed by stronger response recovery in Group Relevant than in Group Irrelevant. In Experiment 2, which included recording of eye-gaze behavior, dwell times on contexts were longer in Group Relevant than in Group Irrelevant. Our results support the idea that relevant contexts receive more attention, leading to stronger context specificity of learning.

Stimuli in our sensory environment differ with respect to their physical salience but moreover may acquire motivational salience by association with reward. If we repeatedly observed that reward is available in the context of a particular cue but absent in the context of another cue the former typically attracts more attention than the latter. However, we also may encounter cues uncorrelated with reward. A cue with 50% reward contingency may induce an average reward expectancy but at the same time induces high reward uncertainty. In the current experiment we examined how both values, reward expectancy and uncertainty, affected overt attention. Two different colors were established as predictive cues for low reward and high reward respectively. A third color was followed by high reward on 50% of the trials and thus induced uncertainty. Colors then were introduced as distractors during search for a shape target, and we examined the relative potential of the color distractors to capture and hold the first fixation. We observed that capture frequency corresponded to reward expectancy while capture duration corresponded to uncertainty. The results may suggest that within trial reward expectancy is represented at an earlier time window than uncertainty.

We conducted a human fear conditioning experiment in which three different color cues were followed by an aversive electric shock on 0, 50, and 100% of the trials, and thus induced low (L), partial (P), and high (H) shock expectancy, respectively. The cues differed with respect to the strength of their shock association (L < P < H) and the uncertainty of their prediction (L < P > H). During conditioning we measured pupil dilation and ocular fixations to index differences in the attentional processing of the cues. After conditioning, the shock-associated colors were introduced as irrelevant distracters during visual search for a shape target while shocks were no longer administered and we analyzed the cues’ potential to capture and hold overt attention automatically. Our findings suggest that fear conditioning creates an automatic attention bias for the conditioned cues that depends on their correlation with the aversive outcome. This bias was exclusively linked to the strength of the cues’ shock association for the early attentional processing of cues in the visual periphery, but additionally was influenced by the uncertainty of the shock prediction after participants fixated on the cues. These findings are in accord with attentional learning theories that formalize how associative learning shapes automatic attention.

In 3 human predictive learning experiments, we investigated whether the allocation of attention can come under the control of contextual stimuli. In each experiment, participants initially received a conditional discrimination for which one set of cues was trained as relevant in Context 1 and irrelevant in Context 2, and another set was relevant in Context 2 and irrelevant in Context 1. For Experiments 1 and 2, we observed that a second discrimination based on cues that had previously been trained as relevant in Context 1 during the conditional discrimination was acquired more rapidly in Context 1 than in Context 2. Experiment 3 revealed a similar outcome when new stimuli from the original dimensions were used in the test stage. Our results support the view that the associability of a stimulus can be controlled by the stimuli that accompany it.