52% Yes, a signiicant crisis 3% No, there is no crisis 7% Don't know 38% Yes, a slight crisis 38% Yes, a slight crisis 1,576 RESEARCHERS SURVEYED M ore than 70% of researchers have tried and failed to reproduce another scientist's experiments, and more than half have failed to reproduce their own experiments. Those are some of the telling figures that emerged from Nature's survey of 1,576 researchers who took a brief online questionnaire on reproducibility in research. The data reveal sometimes-contradictory attitudes towards reproduc-ibility. Although 52% of those surveyed agree that there is a significant 'crisis' of reproducibility, less than 31% think that failure to reproduce published results means that the result is probably wrong, and most say that they still trust the published literature. Data on how much of the scientific literature is reproducible are rare and generally bleak. The best-known analyses, from psychology 1 and cancer biology 2 , found rates of around 40% and 10%, respectively. Our survey respondents were more optimistic: 73% said that they think that at least half of the papers in their field can be trusted, with physicists and chemists generally showing the most confidence. The results capture a confusing snapshot of attitudes around these issues, says Arturo Casadevall, a microbiologist at the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland. "At the current time there is no consensus on what reproducibility is or should be. " But just recognizing that is a step forward, he says. "The next step may be identifying what is the problem and to get a consensus. "

Here we provide further details on the replications, the estimation of standardized effect sizes and complementary replicability indicators, the implementation of the prediction markets and surveys, the comparison of prediction market beliefs, survey beliefs, and replication outcomes, the comparison of reproducibility indicators to experimental economics and the psychological sciences, and additional results and data for the individual studies and markets. The code used for the estimation of replication power, standardized effect sizes, all complementary replication indicators, and all results is posted at OSF (https://osf.io/pfdyw/). Replications Inclusion criteriaWe replicated 21 experimental studies in the social sciences published between 2010 and 2015 in Nature and Science. We included all studies that fulfilled our inclusion criteria for:(i) the journal and time period, (ii) the type of experiment, (iii) the subjects included in the experiment, (iv) the equipment and materials needed to implement the experiment, and (v) the results reported in the experiment. We did not exclude studies that had already been subject to a replication, as this could affect the representativity of the included studies. We define and discuss the five inclusion criteria below. Journal and time period: We included experimental studies published in Nature andScience between 2010 and 2015. The reason for focusing on these two journals is that they are typically considered the two most prestigious general science journals. Articles published in these journals are considered exciting, innovative, and important, which is also reflected in their high impact factors. * Number of observations; number of individuals provided in parenthesis. † Replicated; significant effect (p < 0.05) in the same direction as in original study. ‡ Statistical power to detect 50% of the original effect size r. § Relative standardized effect size. * Belief about the probability of replicating in stage 1 (90% power to detect 75% of the original effect size).† Predicted added probability of replicating in stage 2 (90% power to detect 50% of the original effect size) compared to stage 1. * Mean number of tokens (points) invested per transaction. † Mean number of shares bought or sold per transaction.

Lab-based interventions have been ineffective in changing individuals' implicit racial attitudes for more than brief durations, and exposure to high-status Black exemplars like Obama has proven ineffective in shifting societal-level racial attitudes. Antiracist social movements, however, offer a potential societal-level alternative for reducing racial bias. Racial attitudes were examined before and during Black Lives Matter (BLM) and its high points of struggle with 1,369,204 participants from 2009 to 2016. After controlling for changes in participant demographics, overall implicit attitudes were less pro-White during BLM than pre-BLM, became increasingly less pro-White across BLM, and were less pro-White during most periods of high BLM struggle. Considering changes in implicit attitudes by participant race, Whites became less implicitly pro-White during BLM, whereas Blacks showed little change. Regarding explicit attitudes, Whites became less pro-White and Blacks became less pro-Black during BLM, each moving toward an egalitarian "no preference" position.

Lab-based interventions have been ineffective in changing individuals’ implicit racial attitudes for more than brief durations, and exposure to high-status Black exemplars like Obama has proven ineffective in shifting societal-level racial attitudes. Anti-racist social movements, however, offer a potential societal-level alternative for reducing racial bias. Racial attitudes were examined before and during Black Lives Matter (BLM) and its high points of struggle with 1,369,204 participants from 2009 to 2016. After controlling for changes in participant demographics, overall implicit attitudes were less pro-White during BLM than pre-BLM, became increasingly less pro-White across BLM, and were less pro-White during most periods of high BLM struggle. Considering changes in implicit attitudes by participant race, Whites became less implicitly pro-White during BLM while Blacks showed little change. Regarding explicit attitudes, Whites became less pro-White and Blacks became less pro-Black during BLM, each moving toward an egalitarian “no preference” position.