Why do authors derive new cardiovascular clinical prediction rules in the presence of existing rules? A mixed methods study

Abstract

BackgroundResearchers should examine existing evidence to determine the need for a new study. It is unknown whether developers evaluate existing evidence to justify new cardiovascular clinical prediction rules (CPRs).ObjectiveWe aimed to assess whether authors of cardiovascular CPRs cited existing CPRs, why some authors did not cite existing CPRs, and why they thought existing CPRs were insufficient.MethodDerivation studies of cardiovascular CPRs from the International Register of Clinical Prediction Rules for Primary Care were evaluated. We reviewed the introduction sections to determine whether existing CPRs were cited. Using thematic content analysis, the stated reasons for determining existing cardiovascular CPRs insufficient were explored. Study authors were surveyed via e-mail and post. We asked whether they were aware of any existing cardiovascular CPRs at the time of derivation, how they searched for existing CPRs, and whether they thought it was important to cite existing CPRs.ResultsOf 85 derivation studies included, 48 (56.5%) cited existing CPRs, 33 (38.8%) did not cite any CPR, and four (4.7%) declared there was none to cite. Content analysis identified five categories of existing CPRs insufficiency related to: (1) derivation (5 studies; 11.4% of 44), (2) construct (31 studies; 70.5%), (3) performance (10 studies; 22.7%), (4) transferability (13 studies; 29.5%), and (5) evidence (8 studies; 18.2%). Authors of 54 derivation studies (71.1% of 76 authors contacted) responded to the survey. Twenty-five authors (46.3%) reported they were aware of existing CPR at the time of derivation. Twenty-nine authors (53.7%) declared they conducted a systematic search to identify existing CPRs. Most authors (90.7%) indicated citing existing CPRs was important.ConclusionCardiovascular CPRs are often developed without citing existing CPRs although most authors agree it is important. Common justifications for new CPRs concerned construct, including choice of predictor variables or relevance of outcomes. Developers should clearly justify why new CPRs are needed with reference to existing CPRs to avoid unnecessary duplication.

Predictors for independent external validation of cardiovascular risk clinical prediction rules: Cox proportional hazards regression analyses

“…Although the number of studies reporting CPR research has been rapidly increasing [8, 9, 30], too much focus is still on creating new CPRs rather than externally validating and assessing the impacts of existing CPRs [6, 8, 13]. Cardiovascular risk CPR research has not been an exception [2].…”

“…Researchers interested in developing a new cardiovascular risk CPR should systematically review existing evidence and assess whether a new CPR is needed [13]. When creating a new cardiovascular CPR is clearly justified, it should be created using proper design and rigorous methods to avoid adding redundant CPRs.…”

“…Firstly, we reviewed features of CPR derivation that might be important to researchers planning an external validation [13] and selected the following six characteristics of derivation: study design, geographic location, sample size, number of predictors, presentation format, and validation in derivation. A cohort study is an ideal design when deriving a CPR.…”

Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature

“…Existing CPRs with potential should be updated, validated or tested in an impact study before a new CPR is developed [54, 62, 63]. If a new CPR is derived, researchers should clearly justify why it is required, with reference to existing CPRs, to avoid research waste and duplication of efforts [64]. Qualitative research with clinicians can be useful in determining whether a proposed CPR is clinically relevant, and to assess the credibility of the proposed predictor variables [65, 66].…”

GPs’ familiarity with and use of cardiovascular clinical prediction rules: a UK survey study

“… 6 This may be owing to the following inefficiencies in cardiovascular CPR development. First, many authors do not justify why new CPRs are needed by examining existing CPRs, 7 which often leads to the development of redundant CPRs. Second, most CPRs do not get evaluated in an independent external validation study even many years after they are developed.…”