Creation of a ustekinumab external control arm for Crohn’s disease using electronic health records data: a pilot study

Abstract

ObjectivesThe use of external control arms to study treatment effects is growing in interest among drug sponsors and regulators. However, experience with performing these kinds of studies for complex, immune-mediated diseases is limited. We sought to analyze a retrospective cohort of Crohn’s patients to predict the outcome of a prospective cohort.MethodsWe queried electronic health records databases and screened records at the University of California, San Francisco to identify patients meeting the eligibility criteria of TRIDENT, a concurrent trial involving ustekinumab as a reference arm. Timepoints were defined to balance the tradeoff between missing disease activity and bias. We compared two imputation models by their impacts on cohort membership and outcomes. We compared the results of ascertaining disease activity using structured data algorithms against manual review. We used these data to estimate ustekinumab’s real-world effectiveness.ResultsScreening identified 183 patients. 30% of the cohort had missing baseline data. Two imputation models were tested and had similar effects on cohort definition and outcomes. Algorithms for ascertaining non-symptom-based elements of disease activity were similar in accuracy to manual review. The final cohort consisted of 56 patients. 34% of the cohort was in steroid-free clinical remission by week 24.ConclusionsWe predict that a third of the ustekinumab-treated cohort in TRIDENT will be in steroid-free remission by week 24. However, our prediction is limited by substantial missing data. Efforts to improve real-world data capture and align trial design with clinical practice may enable more robust future studies and improve trial efficiency.STUDY HIGHLIGHTSWHAT IS KNOWNExternal control arm studies are receiving growing interest from drug sponsors and regulators as a potential source of real-world evidenceHowever, the feasibility and robustness of this approach is currently limitedUstekinumab is an FDA-approved treatment of moderately to severely active Crohn’s diseaseWHAT IS NEW HEREWe derived a retrospective cohort of patients designed to resemble the participants of TRIDENT, a concurrent phase 2b trial using ustekinumab as a reference armWe predict that about 34% of the ustekinumab-assigned participants in TRIDENT will be in steroid-free clinical remission by week 24.EHR structured data algorithms may be an accurate and less laborious alternative to manual abstraction of non-symptom-based components of the Crohn’s Disease Activity IndexReal-world practice differs from controlled studies in important ways, including the treatment goals and the timing of encountersThese differences can pose challenges to the feasibility of external control arm studies, and must be addressed to enable this novel study designThe methods, data, and code used in this pilot study are shared here for reproducibility and enhancement by others

Personalizing treatment selection in Crohn’s disease: a meta-analysis of individual participant data from fifteen randomized controlled trials

“…Some degree of selection bias is to be expected of all trials insofar as they contain additional inclusion and exclusion criteria that are not a requirement for receiving clinical care. Indeed, we and others have observed this in the context of comparing RCT- and real-world cohorts 8–10 . What has been unclear, however, is the extent to which these biases may distort treatment outcomes.…”

Real‐world effectiveness of ustekinumab and vedolizumab in TNF‐exposed pediatric patients with ulcerative colitis

“…The timing of clinic visits and data capture in real-world settings commonly differs from those in controlled studies due to a variety of factors, including provider practice patterns, patients' state of illness or health, and social determinants of health impacting the ability to follow-up. 19 ICN is a learning collaborative that disseminates best practice recommendations, 20 but there are no formal guidelines on monitoring pediatric patients on UST or VDZ. Therefore, patients in our study were monitored based on individual center protocols and provider recommendations.…”

Review article: Externally derived control arms—An opportunity for clinical trials in inflammatory bowel disease?

“…39 For example, a recent attempt at creating an ustekinumab external control arm for CD using electronic health records data was severely hindered by missingness. 40…”

“…However, this would be ill‐advised in IBD given the variable quality, lack of homogeneity across most studies and the lack of a RWD core outcome set until recently 39 . For example, a recent attempt at creating an ustekinumab external control arm for CD using electronic health records data was severely hindered by missingness 40 …”