Previous research has shown that capture of visual attention is influenced by prior learning about reward: signals of high reward value are more likely to capture attention (and gaze) than signals of low reward value. In the current study, we show in two experiments that attentional priority is modulated not only by reward magnitude, but also by the uncertainty associated with that reward. Participants completed a visual search task in which they were required to make a saccade to a target shape to earn monetary reward. The color of a color-singleton distractor in the search array signaled the reward outcome(s) that were available. Different distractor colors were associated with different degrees of variance and expected value in reward outcome. Notably, participants were never required to look at the colored distractor, and doing so would slow their response to the target. Nevertheless participants often made eye-movements towards the distractor, and across both experiments they were more likely to look at distractors associated with high outcome variance versus low outcome variance. This pattern was observed when all distractors had equal expected value (Experiment 1), and when the difference in variance was opposed by a substantial difference in expected value (i.e., the high-variance distractor had low expected value, and vice versa: Experiment 2). Our results suggest that reward variance exerts a critical role in modulating rapid attentional and oculomotor priority. More generally, these findings are consistent with theoretical accounts that propose an information-seeking role for visual attention, and suggest that prioritization in line with “attentional exploration” can operate rapidly and on a learned basis.