Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples With Apples

Lodi, Sara; Phillips, Andrew; Lundgren, Jens D.; Logan, Roger; Sharma, Shweta; Cole, Stephen R.; Babiker, Abdel; Law, Matthew; Chu, Haitao; Byrne, Dana; Horban, Andrzéj; Sterne, Jonathan A C; Porter, Kholoud; Sabin, Caroline; Costagliola, Dominique; Abgrall, Sophie; Gill, John; Touloumi, Giota; Pacheco, Antônio Guilherme; Sighem, Ard van; Reiss, Peter; Bucher, Heiner C.; Giménez, Alexandra Montoliu; Jarrín, Inmaculada; Wittkop, Linda; Meyer, Laurence; Pérez‐Hoyos, S.; Justice, Amy C.; Neaton, James D.; Hernán, Miguel A.

doi:10.1093/aje/kwz100

Cited by 89 publications

(79 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thirdly, we emulated a Target Trial [29][30][31] of HIV-infected patients aged ≥16 years already initiated on ART within 14 days of facility-based HIV care enrolment to estimate the causal effect 32 of same-day ART (vs early ART) on the composite unfavourable treatment outcome of death, loss to follow-up (LTFU), viral failure and treatment switching to a second-line ART in the absence of documented viral failure. Time zero was the date of ART initiation because some captured outcomes (viral failure, treatment switch) could only have happened after ART initiation and the outcomes of death and LTFU before ART initiation were not well defined (e.g.…”

Section: Analyses and Main Definitionsmentioning

confidence: 99%

The impact of same-day antiretroviral therapy initiation under the WHO Treat-All policy

Kerschberger

Boulle

Kuwengwa

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: The World Health Organization recommends rapid initiation of antiretroviral therapy (ART) for people living with HIV, with the option to start treatment on the day of diagnosis (same day ART). However, the effect of same day ART remains unknown in realistic public sector settings. Methods: We established a cohort of ≥16 year old patients who initiated first-line ART under Treat-All in Eswatini between 2014-2016, either on the day of HIV care enrolment (same day ART) or 1-14 days thereafter (early ART). Directed acyclic graphs, flexible parametric survival analysis and targeted maximum likelihood estimation (TMLE) were used to estimate the effect of same day ART initiation on the composite unfavourable treatment outcome (loss to follow-up;death;viral failure). Results: Of 1328 patients, 839 (63.2%) initiated same day ART. The adjusted hazard ratio of the unfavourable outcome was increased by 48% (95% CI:1.16-1.89) for same day ART compared with early ART. TMLE suggested that after 1 year, 28.9% of patients would experience the unfavourable outcome under same day ART compared with 21.2% under early ART (difference: 7.7%; 1.3-14.1%). This effect varied over time in regression analysis and TMLE, with a higher hazard during the first year after HIV care enrolment and a similar hazard thereafter. Conclusions: Caution is needed in implementing same day ART in routine settings.

show abstract

Section: Analyses and Main Definitionsmentioning

confidence: 99%

The impact of same-day antiretroviral therapy initiation under the WHO Treat-All policy

Kerschberger

Boulle

Kuwengwa

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Steps 1-3 are applicable to all trial designs using RWD external comparators, and are reminiscent of the steps for the direct comparison of randomised and observational studies outlined by Lodi et al building on the work of Hernan and Robins in emulating a hypothetical clinical trial (i.e. a target trial) [29,30]. We recommend identifying key factors contributing to bias (Step 1), adjusting for baseline characteristics and confounders in analysis (Step 2), as well as modelling and quantifying the potential bias from other key factors, in a process called quantitative bias analysis [QBA] (Step 3).…”

Section: Methodsmentioning

confidence: 99%

A Framework for Methodological Choice and Evidence Assessment for Studies Using External Comparators from Real-World Data

et al. 2020

View full text Add to dashboard Cite

Several approaches have been proposed recently to accelerate the pathway from drug discovery to patient access. These include novel designs such as using controls external to the clinical trial where standard randomised controls are not feasible. In parallel, there has been rapid growth in the application of routinely collected healthcare 'real-world' data for post-market safety and effectiveness studies. Thus, using real-world data to establish an external comparator arm in clinical trials is a natural next step. Regulatory authorities have begun to endorse the use of external comparators in certain circumstances, with some positive outcomes for new drug approvals. Given the potential to introduce bias associated with observational studies, there is a need for recommendations on how external comparators should be best used. In this article, we propose an evaluation framework for real-world data external comparator studies that enables full assessment of available evidence and related bias. We define the principle of exchangeability and discuss the applicability of criteria described by Pocock for consideration of the exchangeability of the external and trial populations. We explore how trial designs using real-world data external comparators fit within the evidence hierarchy and propose a four-step process for good conduct of external comparator studies. This process is intended to maximise the quality of evidence based on careful study design and the combination of covariate balancing, bias analysis and combining outcomes.

show abstract

“…However, like all per-protocol analyses based on standard methods, this analysis could have been subject to bias because the participants who chose to adhere to their treatment might have been systematically different from those who were non-adherent, i.e., with regard to their baseline and post-randomization prognostic factors [7] . Recently, novel methods for causal inference, called g-methods, have been used to estimate the per-protocol effect in randomized controlled trials [8][9][10][11][12] . These methods include inverse probability weighting, the parametric g-formula and g-estimation.…”

Section: Introductionmentioning

confidence: 99%

Per-Protocol Analysis of The ZINC Trial for HIV Disease Among Alcohol Users

Lodi

Freiberg

Gnatienko

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Background. The ZINC trial randomized HIV-positive heavy drinkers to either daily zinc supplementation or placebo. The primary outcome was change in VACS index, a predictor of mortality, between baseline and 18 months. Because adherence and follow-up were suboptimal, the intention-to-treat analysis, which was not statistically significant, may have underestimated the effect of the zinc supplementation. Objective. We estimated the per-protocol effect of zinc versus placebo in the ZINC trial (i.e., the effect that would have been observed if all participants had had high adherence and none were lost to follow-up).Methods. Adherence was measured as the self-reported percentage of pills taken in the previous 6 weeks and assessed at all post-baseline visits. We used inverse probability weighting to estimate and compare the change in VACS index at 18 months in the zinc and placebo groups had all the trial participants had high adherence (i.e., cumulative adherence ≥80% at 18 months). To examine trends by level of adherence, we rerun the analyses using thresholds for high-adherence of 70% and 90% of average self-reported pill coverage.Results. The estimated (95% confidence interval) change in VACS index was –2.16 (-8.07, 3.59) and 5.84 (0.73, 11.80) under high adherence and no loss to follow-up in the zinc and placebo groups, respectively. The per-protocol effect estimate of the mean difference in change between the zinc and placebo group was -8.01 (-16.42, 0.01), somewhat larger than the intention-to-treat effect difference in change (-4.68 (-9.62, 0.25)), but it was still not statistically significant. The mean difference in change between individuals in the zinc and placebo group was -4.07 (-11.5,2.75) and -12.34 (-20.14,-4.14) for high adherence defined as 70% and 90% of pill-coverage, respectively. Conclusions. Overall, high adherence to zinc was associated with lower VACS score, but confidence intervals were wide and crossed 0. Further studies with larger sample size are needed to quantify the benefits of zinc supplementation in this population.Trial Registration. ClinicalTrials.gov Identifier: NCT01934803, August 30, 2013

show abstract

Effect Estimates in Randomized Trials and Observational Studies: Comparing Apples With Apples

Cited by 89 publications

References 44 publications

The impact of same-day antiretroviral therapy initiation under the WHO Treat-All policy

The impact of same-day antiretroviral therapy initiation under the WHO Treat-All policy

A Framework for Methodological Choice and Evidence Assessment for Studies Using External Comparators from Real-World Data

Per-Protocol Analysis of The ZINC Trial for HIV Disease Among Alcohol Users

Contact Info

Product

Resources

About