All adaptive organisms face the fundamental tradeoff between pursuing a known reward (exploitation) and sampling lesser-known options in search of something better (exploration). Theory suggests at least two strategies for solving this dilemma: a directed strategy in which choices are explicitly biased toward information seeking, and a random strategy in which decision noise leads to exploration by chance. In this work we investigated the extent to which humans use these two strategies. In our “Horizon task,” participants made explore– exploit decisions in two contexts that differed in the number of choices that they would make in the future (the time horizon). Participants were allowed to make either a single choice in each game (horizon 1), or 6 sequential choices (horizon 6), giving them more opportunity to explore. By modeling the behavior in these two conditions, we were able to measure exploration-related changes in decision making and quantify the contributions of the two strategies to behavior. We found that participants were more information seeking and had higher decision noise with the longer horizon, suggesting that humans use both strategies to solve the exploration– exploitation dilemma. We thus conclude that both information seeking and choice variability can be controlled and put to use in the service of exploration.

When making decisions on the basis of past experiences, people must rely on their memories. Human memory has many well-known biases, including the tendency to better remember highly salient events. We propose an extreme-outcome rule, whereby this memory bias leads people to overweight the largest gains and largest losses, leading to more risk seeking for relative gains than for relative losses. To test this rule, in two experiments, people repeatedly chose between fixed and risky options, where the risky option led equiprobably to more or less than did the fixed option. As was predicted, people were more risk seeking for relative gains than for relative losses. In subsequent memory tests, people tended to recall the extreme outcome first and also judged the extreme outcome as having occurred more frequently. Across individuals, risk preferences in the risky-choice task correlated with these memory biases. This extreme-outcome rule presents a novel mechanism through which memory influences decision making.

When faced with risky decisions, people tend to be risk averse for gains and risk seeking for losses (the reflection effect). Studies examining this risk-sensitive decision making, however, typically ask people directly what they would do in hypothetical choice scenarios. A recent flurry of studies has shown that when these risky decisions include rare outcomes, people make different choices for explicitly described probabilities than for experienced probabilistic outcomes. Specifically, rare outcomes are overweighted when described and underweighted when experienced. In two experiments, we examined risk-sensitive decision making when the risky option had two equally probable (50%) outcomes. For experience-based decisions, there was a reversal of the reflection effect with greater risk seeking for gains than for losses, as compared to description-based decisions. This fundamental difference in experienced and described choices cannot be explained by the weighting of rare events and suggests a separate subjective utility curve for experience.

Whether buying stocks or playing the slots, people making real-world risky decisions often rely on their experiences with the risks and rewards. These decisions, however, do not occur in isolation but are embedded in a rich context of other decisions, outcomes, and experiences. In this paper, we systematically evaluate how the local context of other rewarding outcomes alters risk preferences. Through a series of four experiments on decisions from experience, we provide evidence for an extreme-outcome rule, whereby people overweight the most extreme outcomes (highest and lowest) in a given context. As a result, people should be more risk seeking for gains than losses, even with equally likely outcomes. Across the experiments, the decision context was varied so that the same outcomes served as the high extreme, low extreme, or neither. As predicted, people were more risk seeking for relative gains, but only when the risky option potentially led to the high-extreme outcome. Similarly, people were more risk averse for relative losses, but only when the risky option potentially led to the low-extreme outcome. We conclude that in risky decisions from experience, the biggest wins and the biggest losses seem to matter more than they should.