Why Does Primary Angioplasty Not Work in Registries? Quantifying the Susceptibility of Real-World Comparative Effectiveness Data to Allocation Bias

Abstract

The online-only Data Supplement is available at http://circoutcomes.ahajournals.org/lookup/suppl/ Background-Meta-analysis of registries (comparative effectiveness research) shows that primary angioplasty and fibrinolysis have equivalent real-world survival. Yet, randomized, controlled trials consistently find primary angioplasty superior. Can unequal allocation of higher-risk patients in registries have masked primary angioplasty benefit? Methods and Results-First, we constructed a model to demonstrate the potential effect of allocation bias. We then analyzed published registries (55 022 patients) for allocation of higher-risk patients (Killip class ≥1) to determine whether the choice of reperfusion therapy was affected by the risk level of the patient. Meta-regression was used to examine the relationship between differences in allocation of high-risk patient to primary angioplasty or fibrinolysis and mortality. Initial modeling suggested that registry outcomes are sensitive to allocation bias of high-risk patients. Across the registries, the therapy receiving excess high-risk patients had worse mortality. Unequal distribution of high-risk status accounted for most of the between-registry variance (adjusted R 2 meta =83.1%). Accounting for differential allocation of higher-risk patients, primary angioplasty gave 22% lower mortality (odds ratio, 0.78; 95% confidence interval, 0.64-0.97; P=0.029). We derive a formula, called the number needed to abolish, highlighting situations in which comparative effectiveness studies are particularly vulnerable to this bias. Conclusions-In ST-segment elevation myocardial infarction, clinicians' preference for management of a few high-risk patients can shift mortality substantially. Comparative effectiveness research in any disease is vulnerable to this, especially diseases with an immediately identifiable high-risk subgroup that clinicians prefer to allocate to 1 therapy. For this reason, preliminary indications from registry-based comparative effectiveness research should be definitively tested by randomized, controlled trials. (Circ Cardiovasc Qual Outcomes. 2012;5:759-766.)

A Randomized Trial to Assess Catheter Ablation Versus Rate Control in the Management of Persistent Atrial Fibrillation in Heart Failure

“…We do not know the greater denominator beyond the 101 who were referred to the trial; many patients might have not been referred because of geography, lack of symptoms, aversion to invasive procedures, or disinclination to volunteer. Although randomized trials have weaknesses, they remain the gold standard for evaluating therapeutic choices (37).…”

Additional functional outcomes after endovascular treatment for intermittent claudication

“…A possible selection bias is the inclusion criteria based on the definition of severity (CLI or IC) made by the vascular surgeon and local preferences for treatment, as only EVT was included. Registry outcome and adverse event rates are sensitive to allocation bias for high-risk patients, as shown in cardiac registries (24). This is of special importance when different treatments are compared in registries, e.g.…”

Difficulty in detecting discrepancies in a clinical trial report: 260-reader evaluation

“…Critical appraisal of an article involves far more important aspects than detecting discrepancies. For example, readers should be aware of the limitations of making therapeutic decisions based on observational comparisons 22 , 23 and be aware of the need for appropriate statistical testing. However, when readers note discrepancies in a trial, it would be helpful if they made them available to other readers because most readers will not notice most discrepancies in the normal course of events.…”