Increasing value and reducing waste in research design, conduct, and analysis

Abstract

Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

A primary care Web-based Intervention Modeling Experiment replicated behavior changes seen in earlier paper-based experiment

“…• The IME methodology may potentially be considered as a way of developing theory-based behaviour change interventions prior to evaluation in a full-scale 8 …”

“…Recruitment was done in two stages, reflecting the stages of an IME [8]. The first stage recruited GPs to complete an online questionnaire comprising 20 questions about antibiotic prescribing behaviour, eight clinical scenarios that required antibiotic prescribing decisions and four general questions about the GP's background.…”

“…We have also evaluated a new behaviour change intervention in a randomised trial and found that it changed behavioural simulation as expected based on its theory-based design. Replication studies are an important part of increasing value and reducing waste in research [8] and this replication study gives us greater confidence in the IME methodology than a single Table and figure legends Table 1 Baseline characteristics. were GPs who were not in the first stage but who were invited because we were unsure that 150 target GPs would respond, which turned out to be correct.…”

“…In an IME key elements of the intervention are delivered, using a randomised design, in a manner that approximates the real world but where the measured outcome is generally an interim outcome, a proxy for the behaviour of interest. Although we thought the methodology promising, there had been no 9 of 21 A CCEPTED MANUSCRIPT replication study, which we considered essential if other investigators were to have confidence in the methodology; a single success is rarely sufficiently compelling to support widespread adoption [8].…”

Explanation and Elaboration Document for the STROBE‐Vet Statement: Strengthening the Reporting of Observational Studies in Epidemiology—Veterinary Extension

“…Some omissions can seriously limit the utility of the research by either hiding limitations or creating unwarranted doubt about the studies’ conclusions. These omissions, in turn, increase research wastage 8, 9, 10, 11, 12, 13. Study results are usually used by people other than the manuscript authors to make decisions.…”