What meta‐research has taught us about research and changes to research practices

Abstract

Meta‐research has become increasingly popular and has provided interesting insights on what can go well and what can go wrong with research practices and scientific studies. Many stakeholders are taking actions to try to solve problems and biases identified through meta‐research. However, very often there is little or no evidence that specific recommendations and actions may actually lead to improvements and a favorable benefit‐harm ratio. The current commentary offers an eclectic overview of what we have learned from meta‐research efforts (mostly observational, but also some quasi‐experimental and experimental work) and what the implications of this evidence may be for changing research practices. Areas discussed include the study (and differentiation) of genuine effects and biases, fraud (including the impact of new technologies), peer review, replication and reproducibility checks, transparency indicators, and the interface of research practices with reward systems. Meta‐research has offered on all of these fronts empirical evidence that sometimes pertains even to large effects of extreme biases. Continued surveys of research practices and results may offer timely updates of the status of research and its biases, as these may change markedly over time. Meta‐research should be seen as part of research, not separate from it, in their concurrent evolution.