Accelerating Psychological Science With Metastudies: A Demonstration Using the Risky-Choice Framing Effect

Abstract

A metastudy is a set of many tiny studies (microstudies) created from a much larger collection of possibilities. Metastudies can yield many of the benefits of time-consuming replications and meta-analyses but more efficiently and with greater attention to generalizability and the causal effects of moderators. Statistical precision and power are higher than in studies with the same total sample size but with fewer conditions and more participants per condition. In this article, we describe metastudies and their benefits, demonstrate how to conduct a metastudy using the well-known risky-choice framing effect as an example, and report on the generalizability of that effect. In three metastudies (total N = 2,338), the framing effect generalized well across most of the potential moderators tested, as was expected. Surprisingly, however, the effect was up to twice as large when the certain option was replaced with a slightly risky option; prospect theory predicts the opposite, and fuzzy-trace theory predicts no difference. Metastudies provide a relatively quick and not-so-painful way of examining an effect’s generalizability without waiting for a meta-analysis. Both individual labs and multilab networks are encouraged to shift from traditional studies to metastudies.

The Risk-Taking Software Engineer: A Framed Portrait

“…They studied different wordings (with the same expected outcome) and found that choices described as losses induce higher risk-taking than choices described as gains. These results have since been replicated in various studies [6]- [10].…”

The cognitive load effect in working memory: Refreshing the empirical landscape, removing outdated explanations

“…Therefore, Experiment 3 differed from the preceding experiments not only with respect to the manipulations for testing our two predictions, but also with respect to a number of ancillary features, such as the choice of the processing task, the stimuli, and the timing. This variance between experiments on dimensions that we expect to be immaterial is not a weakness but a strength of the present study, as it broadens the generalizability of our findings (Baribault et al, 2018;DeKay, et al, 2022).…”

People conform to social norms when gambling with lives or money

“…We suspect that participants were more concerned with the potential catastrophic outcomes with pandas (an endangered animal) than with money, so they were more prone to avoid gambling with pandas' lives (since choosing the certain option ensures that only 10 pandas are lost, or at least 10 pandas are saved). This speculation should be addressed in www.nature.com/scientificreports/ future research, especially because the original version of ADP task did not observe an unusually high degree of risk aversion when dealing with human lives 21,42 .…”

“…Similarly, studies that compared real and hypothetical financial decisions suggests that real choices are sometimes associated with more loss aversion, especially when stakes are high 55,56 . It is worth noting though that the Asian Disease Paradigm (ADP) used in the current research has been very successful in identifying mechanisms underlying financial and non-financial choices across multiple domains, and in the real world 42,46,57,58 . Using ADP to understand social conformity can help with designing more effective public policy whenever following social norms might be desirable or undesirable.…”