Objective To investigate the agreement between direct and indirect comparisons of competing healthcare interventions.Design Meta-epidemiological study based on sample of meta-analyses of randomised controlled trials. Data sources Cochrane Database of Systematic Reviews and PubMed.Inclusion criteria Systematic reviews that provided sufficient data for both direct comparison and independent indirect comparisons of two interventions on the basis of a common comparator and in which the odds ratio could be used as the outcome statistic.Main outcome measure Inconsistency measured by the difference in the log odds ratio between the direct and indirect methods. ResultsThe study included 112 independent trial networks (including 1552 trials with 478 775 patients in total) that allowed both direct and indirect comparison of two interventions. Indirect comparison had already been explicitly done in only 13 of the 85 Cochrane reviews included. The inconsistency between the direct and indirect comparison was statistically significant in 16 cases (14%, 95% confidence interval 9% to 22%). The statistically significant inconsistency was associated with fewer trials, subjectively assessed outcomes, and statistically significant effects of treatment in either direct or indirect comparisons. Owing to considerable inconsistency, many (14/39) of the statistically significant effects by direct comparison became non-significant when the direct and indirect estimates were combined. ConclusionsSignificant inconsistency between direct and indirect comparisons may be more prevalent than previously observed. Direct and indirect estimates should be combined in mixed treatment comparisons only after adequate assessment of the consistency of the evidence. IntroductionRandomised controlled trials to compare competing interventions are often lacking, and this situation is unlikely to improve in the future because of the inevitable tension between the high cost of clinical trials and the continuing introduction of new treatments.1 2 The dearth of evidence from head to head randomised controlled trials has led to increased use of indirect comparison methods to estimate the comparative effects of treatment. [1][2][3][4] Indirect comparison of competing interventions can be generally defined as a comparison of different treatments for a clinical indication by using data from separate randomised controlled trials, in contrast to direct comparison within randomised controlled trials. Indirect comparison based on a common comparator can preserve certain strengths of randomised allocation of patients for estimating comparative effects of treatment.1 5 6 The term "adjusted indirect comparison" is used to refer to this indirect comparison based on a common intervention (fig 1). 7 Mixed treatment comparison (also known RESEARCHas network meta-analysis or multiple treatment meta-analysis) is a more complex method that combines both indirect and direct estimates simultaneously. [8][9][10] The validity of indirect and mixed treatment comparisons dep...

BackgroundThe validity of research synthesis is threatened if published studies comprise a biased selection of all studies that have been conducted. We conducted a meta-analysis to ascertain the strength and consistency of the association between study results and formal publication.MethodsThe Cochrane Methodology Register Database, MEDLINE and other electronic bibliographic databases were searched (to May 2009) to identify empirical studies that tracked a cohort of studies and reported the odds of formal publication by study results. Reference lists of retrieved articles were also examined for relevant studies. Odds ratios were used to measure the association between formal publication and significant or positive results. Included studies were separated into subgroups according to starting time of follow-up, and results from individual cohort studies within the subgroups were quantitatively pooled.ResultsWe identified 12 cohort studies that followed up research from inception, four that included trials submitted to a regulatory authority, 28 that assessed the fate of studies presented as conference abstracts, and four cohort studies that followed manuscripts submitted to journals. The pooled odds ratio of publication of studies with positive results, compared to those without positive results (publication bias) was 2.78 (95% CI: 2.10 to 3.69) in cohorts that followed from inception, 5.00 (95% CI: 2.01 to 12.45) in trials submitted to regulatory authority, 1.70 (95% CI: 1.44 to 2.02) in abstract cohorts, and 1.06 (95% CI: 0.80 to 1.39) in cohorts of manuscripts.ConclusionDissemination of research findings is likely to be a biased process. Publication bias appears to occur early, mainly before the presentation of findings at conferences or submission of manuscripts to journals.

BackgroundGeneral practices in the UK contract with the government to receive additional payments for highquality primary care. Little is known about the resulting impact on population health. AimTo estimate the potential reduction in population mortality from implementation of the pay-forperformance contract in England. Design of studyCross-sectional and modelling study. SettingPrimary care in England. MethodTwenty-five clinical quality indicators in the contract had controlled trial evidence of mortality benefit. This was combined with condition prevalence, and the differences in performance before and after contract implementation, to estimate the potential mortality reduction per indicator. Improvement was adjusted for pre-existing trends where data were available. ResultsThe 2004 contract potentially reduced mortality by 11 lives per 100 000 people (lower-upper estimates 7-16) over 1 year, as performance improved from baseline to the target for full incentive payment. If all eligible patients were treated, over and above the target, 56 (29-81) lives per 100 000 might have been saved. For the 2006 contract, mortality reduction was effectively zero, because new baseline performance for a typical practice had already exceeded the target performance for full payment. ConclusionThe contract may have delivered substantial health gain, but potential health gain was limited by performance targets for full payment being set lower than typical baseline performance. Information on both baseline performance and population health gain should inform decisions about future selection of indicators for pay-for-performance schemes, and the level of performance at which full payment is triggered. Keywordsphysician incentive plans; primary health care; quality indicators; quality of health care.