No abstract

SummaryData analysis workflows in many scientific domains have become increasingly complex and flexible. To assess the impact of this flexibility on functional magnetic resonance imaging (fMRI) results, the same dataset was independently analyzed by 70 teams, testing nine ex-ante hypotheses. The flexibility of analytic approaches is exemplified by the fact that no two teams chose identical workflows to analyze the data. This flexibility resulted in sizeable variation in hypothesis test results, even for teams whose statistical maps were highly correlated at intermediate stages of their analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Importantly, meta-analytic approaches that aggregated information across teams yielded significant consensus in activated regions across teams. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset. Our findings show that analytic flexibility can have substantial effects on scientific conclusions, and demonstrate factors related to variability in fMRI. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for multiple analyses of the same data. Potential approaches to mitigate issues related to analytical variability are discussed.

Forty years ago, prospect theory introduced the notion that risky options are evaluated relative to their recent context, causing a significant shift in the study of risky monetary decision-making in psychology, economics, and neuroscience. Despite the central role of past experiences, it remains unclear whether, how, and how much past experiences quantitatively influence risky monetary choices moment-tomoment in a nominally learning-free setting. We analyzed a large dataset of risky monetary choices with trial-by-trial feedback to quantify how past experiences, or recent events, influence risky choice behavior and the underlying processes. We found larger recent outcomes both negatively influence subsequent risk-taking and positively influence the weight put on potential losses. Using a hierarchical Bayesian framework to fit a modified version of prospect theory, we demonstrated that the same risks will be evaluated differently given different past experiences. The computations underlying risky decision-making are fundamentally dynamic, even if the environment is not. Prospect theory is conceivably the most successful recent theory in risky decision-making 1. The theory is founded on the idea that people evaluate options relative to "the past and present context of experience" (p. 277) 1. While this central insight led to two of the most successful components of prospect theory, the reflection effect in risk aversion (diminishing marginal utility of values expressed as concavity for gains and convexity for losses) and loss aversion (overweighting losses compared to gains of the same magnitude), it is unclear how recent events quantitatively influence risky monetary decision-making, moment-to-moment. We sought to quantify how recent events (e.g. previous outcomes) influence choices by analyzing a large dataset of risky choices in the presence of feedback compiled from four studies 2-5. Laboratory and field studies have suggested that recent events shape the subjective value of risky choice options 6-9 and actions 10. The directionality and mechanism supporting these effects are unclear as some studies of risky decision-making in the presence of feedback have found that previous gains, relative to losses, lead to less risk-taking 6,10 or a mixture of less and more risk-taking 7-9,11 , discrepancies that may be related to how and when outcomes are realized, or paid 12. Theories that have sought to specify the directionality of the effect of feedback on risky choice have not described the size of the effect, its characteristics (e.g. how far it extends in time) or the precise decision processes that would be altered by feedback 13. Interpreting these previous findings is complicated by the fact that almost all of these studies included design features intended to elicit temporal context effects (e.g. explicitly-signaled contexts, displays with cumulative elements) and thus may have strengthened, or even created, dynamic context effects. While such an approach identifies an upper bound on the possible presence and ma...

No abstract

Context-dependence is fundamental to risky monetary decision-making. A growing body of evidence suggests that temporal context, or recent events, alters risk-taking at a minimum of three timescales: immediate (e.g. trial-by-trial), neighborhood (e.g. a group of consecutive trials), and global (e.g. task-level). To examine context effects, we created a novel monetary choice set with intentional temporal structure in which option values shifted between multiple levels of value magnitude (“contexts”) several times over the course of the task. This structure allowed us to examine whether effects of each timescale were simultaneously present in risky choice behavior and the potential mechanistic role of arousal, an established correlate of risk-taking, in context-dependency. We found that risk-taking was sensitive to immediate, neighborhood, and global timescales, increasing following small (vs. large) outcome amounts, large positive (but not negative) shifts in context, and when cumulative earnings exceeded expectations. We quantified arousal with skin conductance responses, which were specifically related to the global timescale, increasing with cumulative earnings, suggesting that physiological arousal captures a task-level assessment of performance. We complimented this correlational analysis with a secondary reanalysis of risky monetary choices following the double-blind administration of propranolol and a placebo during a temporally unstructured choice task. We replicated our behavioral finding that risk-taking is context-sensitive at three timescales but found no change in temporal context-effects following propranolol administration. Our results demonstrate that risky decision-making is consistently dynamic at multiple timescales and that arousal is likely the consequence, rather than the cause, of temporal context in risky monetary decision-making.