Estimating treatment effect via simple cross design synthesis

Abstract

Randomized controlled trials (RCTs) are the traditional gold standard evidence for medical decision-making. However, protocols that limit enrollment eligibility introduce selection error that severely limits a RCT's applicability to a wide range of patients. Conversely, high quality observational data can be representative of entire populations, but freedom to choose treatment can bias estimators based on this data. Cross design synthesis (CDS) is an approach to combining both RCT and observational data in a single analysis that capitalizes on the RCT's strong internal validity and the observational study's strong external validity. We proposed and assessed a simple estimator of effect size based on the CDS approach. We evaluated its properties within a formal framework of causal estimation and compared our estimator with more traditional estimators based on single sources of evidence. We show that under ideal conditions the simple CDS estimator is unbiased whenever the observational data-based estimators' treatment selection error is constant across those who are and are not eligible for RCT participation. Whereas this assumption may not often hold in practice, assumptions required for the unbiasedness of usual single-source estimators may also be implausible. We show that, under some reasonable data assumptions, our simple CDS estimator has smaller bias and better coverage than commonly used estimates based on randomized or observational studies alone.

Combining randomized and non‐randomized evidence in clinical research: a review of methods and applications

“…A modern view of the CDS was recently given by Kaizar (), where the statistical framework proposed by Imai et al . () is used to evaluate the statistical properties of a CDS estimator.…”

Age-Dependent Effect of β-Blockers in Preventing Vasovagal Syncope

“…The concept of complex synthesis (ie, combining results across different study types) has received much attention recently in the development of statistical methodology. [38][39][40] Applications of this methodology have been published for cardiovascular events, 41 and complex synthesis is recommended for the assessment of adverse events. 42 The analysis presented here is a simple example of a metaanalysis across different study types but with only 1 study within each study type.…”

The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias

“…But, combining RCT and observational data could be very useful and important for providing greater external validity to the results of RCTs. In fact, our proposed estimators can be easily incorporated into simple cross design synthesis analyses [10] to improve estimation of PATE for population-wide regulation and the estimation of SPATE for subgroup decision-making.…”

“…That is, there are two sets of sufficient conditions for an IPW-based estimator of generalizability bias (i.e., the generalizability bias estimators based on the IPW and DR estimators) to be unbiased:

$\widehat{\Delta }\left(I\right)$ and $\widehat{\Delta }\left(E\right)$ consistent (see Table 1)

Bias of $\widehat{\Delta }\left(I\right)=\text{Bias if}\widehat{\Delta }\left(E\right)$, which occurs whenever the following three conditions are met:

I ⫫ U | T , X ,

I ⫫ U / X (or I ⫫ T | U , X , or T ⫫ U | I , X for binary U).

E [ Y | T, I, X, U ] = g 1 ( T, I, X ) + g 2 ( U, X ),

where the notation X ⫫ Y | Z indicates the independence of X and Y conditional on Z . Conditions (B) are suggested by Li, et al [22] and Kaizar (2011) [10], who considers the bias of $\hat{\gamma }$ in the case of the simple estimator. When the unmeasured variable U is binary, condition (B2) can be replaced with identical condition (B2*) I ⫫ T / U , X or (B2**) T ⫫ U | I , X .…”

“…Several authors have proposed methods for incorporating bias into estimates of population average treatment effects (PATEs) [9, 10, 11]. Our methods provide empirical evidence for the magnitude of these biases that can be directly utilized in these types of estimators.…”