Comparison of Clinical Trial Changes in Primary Outcome and Reported Intervention Effect Size Between Trial Registration and Publication

Abstract

Key Points Question Does primary outcome change between trial registration and publication alter a randomized clinical trial’s reported intervention effect size? Findings In this cross-sectional study that included 389 trials, 130 of them had at least 1 primary outcome change between registration and publication. This significantly overestimated the reported intervention effect size by 16% compared with those without primary outcome change. Meaning Inconsistencies between registered and published primary outcomes in clinical trials are common, with a larger reported intervention effect size among those with primary outcome change than those without.

Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomised trials

“…If investigators choose the method of analysis based on trial data in order to obtain more favourable results (often referred to as 'p-hacking'), this can cause bias [7,8]. Selective reporting has been identified previously, where outcomes with more favourable results are more likely to be reported than other outcomes [9][10][11][12][13][14][15][16][17][18][19][20]. There is some evidence to suggest this may also be a concern for statistical analyses; pre-specification of proposed methods in protocols is often poor, discrepancies between protocols and publications are common, and in some instances, changes may have been made to obtain specific results [5,9,11,14,[21][22][23][24].…”

UK deal over inclisiran

“…This composite outcome, which was not listed among primary or secondary outcomes in the trial’s protocol (ClinicalTrials.gov NCT03400800), included cardiac death, non-fatal myocardial infarction, and stroke as well as an imprecisely defined component described as “any signs or symptoms of cardiac arrest.” Information allowing appraisal of the validity of these results is also unavailable. In general, reported outcomes not listed in trial protocols have larger effect sizes than outcomes identified in advance 8. Although reductions in LDL cholesterol were listed as the primary and secondary outcomes in the trial protocol, these are surrogate endpoints—not all cholesterol lowering drugs have lived up to initial expectations of disease prevention and lower mortality 89…”

“…In general, reported outcomes not listed in trial protocols have larger effect sizes than outcomes identified in advance 8. Although reductions in LDL cholesterol were listed as the primary and secondary outcomes in the trial protocol, these are surrogate endpoints—not all cholesterol lowering drugs have lived up to initial expectations of disease prevention and lower mortality 89…”

Mapping outcome reporting bias in published non-pharmacological randomized trials: a protocol for a cross-sectional study

“…Because of this, an outcome may be considered only fully specified when all 5 elements are present (10). Given the five possible elements that constitute an outcome and a difference in criteria required by international standards (3 elements), it can be considered an adequately reported outcome one that is partially defined (with a domain, measure and time point) or even fully specified (if all 5 possible elements are present) being defined in the registry in a retrospective manner (6,8,9,(11)(12)(13)(14)(15)(16). In this example, "mean change from baseline of Beck Depression Inventory score 12-month post randomization", stated in the primary outcome section in a registry (done prospectively -before the initiation of the clinical trial) in clinicaltrials.gov platform (or other platform) with a registry number (13), would be considered a fully specified outcome (10) and hence adequately reported (13).…”