Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

No abstract

There is increasing interest in utilizing novel markers of cardiovascular disease risk, and consequently, there is a need to assess the value of their use. This scientific statement reviews current concepts of risk evaluation and proposes standards for the critical appraisal of risk assessment methods. An adequate evaluation of a novel risk marker requires a sound research design, a representative at-risk population, and an adequate number of outcome events. Studies of a novel marker should report the degree to which it adds to the prognostic information provided by standard risk markers. No single statistical measure provides all the information needed to assess a novel marker, so measures of both discrimination and accuracy should be reported. The clinical value of a marker should be assessed by its effect on patient management and outcomes. In general, a novel risk marker should be evaluated in several phases, including initial proof of concept, prospective validation in independent populations, documentation of incremental information when added to standard risk markers, assessment of effects on patient management and outcomes, and ultimately, cost-effectiveness.

No abstract