The Brazilian Reproducibility Initiative

Abstract

Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, even though science is usually assessed and funded on a regional or national basis. Here we describe a project to assess the reproducibility of findings in biomedical science published by researchers based in Brazil. The Brazilian Reproducibility Initiative is a systematic, multicenter effort to repeat between 60 and 100 experiments: the project will focus on a set of common methods, repeating each experiment in three different laboratories from a countrywide network. The results, due in 2021, will allow us to estimate the level of reproducibility of biomedical science in Brazil, and to investigate what aspects of the published literature might help to predict whether a finding is reproducible.

Addressing selective reporting of experiments through predefined exclusion criteria

“…As coordinators of the Brazilian Reproducibility Initiative, a multicenter replication of 60-100 experiments from the Brazilian biomedical literature over the last 20 years, conducted by a team of more than 60 labs, we have been faced with the need for validation criteria in many stages during protocol development (Amaral et al, 2019). As the project is meant to be confirmatory in nature, we intend to preregister every protocol, including the analysis plan.…”

“…Here we make the case that predefined inclusion and exclusion criteria for experiments can help solve this conundrum, and discuss ways in which they can be implemented in the workflow of experimental projects, particularly in those of a confirmatory nature. We also describe how we are taking this approach in the Brazilian Reproducibility Initiative, a large-scale multicenter replication of experimental findings in basic biomedical science (Amaral et al, 2019).…”

Addressing selective reporting of experiments – the case for predefined exclusion criteria

“…As coordinators of the Brazilian Reproducibility Initiative (Amaral et al, 2019), a multicenter replication of 60-100 experiments of the Brazilian biomedical literature over the last 20 years conducted by a team of more than 60 labs, we have been faced with the need for validation criteria in many stages during protocol development. As the project is meant to be confirmatory in nature, we intend to preregister every protocol, including the analysis plan.…”

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

“…Detailed protocols and rationale for the Initiative can be found in the Open Science Framework (https://osf.io/6av7k/), as well as in previous publications. 16 …”

“…This might have biased our sample towards articles with less international collaboration, but helped to select findings that were likely to have been obtained in a Brazilian laboratory. In order to have as representative a sample as possible, we performed full-text searches for our methods of interest in a random sample of life sciences articles from the Web of Science 16 (see also https://osf.io/57f8s/). Nevertheless, included experiments were later filtered for those that could be performed with the expertise and infrastructure of our collaborating labs, as well as within our budget.…”