Summarising salient information on historical controls: A structured assessment of validity and comparability across studies

Abstract

Background While placebo-controlled randomised controlled trials remain the standard way to evaluate drugs for efficacy, historical data are used extensively across the development cycle. This ranges from supplementing contemporary data to increase the power of trials to cross-trial comparisons in estimating comparative efficacy. In many cases, these approaches are performed without in-depth review of the context of data, which may lead to bias and incorrect conclusions. Methods We discuss the original ‘Pocock’ criteria for the use of historical data and how the use of historical data has evolved over time. Based on these factors and personal experience, we created a series of questions that may be asked of historical data, prior to their use. Based on the answers to these questions, various statistical approaches are recommended. The strategy is illustrated with a case study in colorectal cancer. Results A number of areas need to be considered with historical data, which we split into three categories: outcome measurement, study/patient characteristics (including setting and inclusion/exclusion criteria), and disease process/intervention effects. Each of these areas may introduce issues if not appropriately handled, while some may preclude the use of historical data entirely. We present a tool (in the form of a table) for highlighting any such issues. Application of the tool to a colorectal cancer data set demonstrates under what conditions historical data could be used and what the limitations of such an analysis would be. Conclusion Historical data can be a powerful tool to augment or compare with contemporary trial data, though caution is required. We present some of the issues that may be considered when involving historical data and what (if any) statistical approaches may account for differences between studies. We recommend that, where historical data are to be used in analyses, potential differences between studies are addressed explicitly.

Incorporating historical controls in clinical trials with longitudinal outcomes using the modified power prior

“…Similarly, researchers who are interested in incorporating historical control arm in the current analysis using historical borrowing methods should also evaluate the compatibility between historical and current data beforehand. The criteria proposed by Pocock are common choices to accomplish the evaluation 28,34 . The results of the real‐data analysis in this study were in accordance with those in the simulation study, and the conditional MPP borrowed the most among the dynamic borrowing methods and the commensurate prior borrowed the least.…”

“…The criteria proposed by Pocock are common choices to accomplish the evaluation. 28,34 The results of the real-data analysis in this study were in accordance with those in the simulation study, and the conditional MPP borrowed the most among the dynamic borrowing methods and the commensurate prior borrowed the least. None of the dynamic borrowing methods find clear evidence that DHA supplementation slows the rate of cognitive decline in patients with Alzheimer's disease.…”

“…In scenarios with moderate or high between‐study heterogeneity, the conditional MPP leads to an inflated type I error rate, whereas the marginal MPP has type I error rate close to 5% and higher statistical power with that of “No borrowing”. Therefore it is desirable to assess the between‐study heterogeneity before implementing the MPP to preclude a highly heterogeneous historical control arm using specific assessment tools 28 . Moreover, a more skeptical prior than the Beta (1, 2) prior used in the sensitivity analysis for the power parameter may be considered to control the type I error rate, which implies that the MPP especially the conditional MPP is excessively liberal in incorporating the historical information.…”

“…Therefore it is desirable to assess the between-study heterogeneity before implementing the MPP to preclude a highly heterogeneous historical control arm using specific assessment tools. 28 Moreover, a more skeptical prior than the Beta (1, 2) prior used in the sensitivity analysis for the power parameter may be considered to control the type I error rate, which implies that the MPP especially the conditional MPP is excessively liberal in incorporating the historical information. Although inflation of the type I error rate is common in the context of historical borrowing methods, regulatory authorities such as FDA do not totally forbid the use of historical borrowing methods due to strict control of the type I error rate.…”

The power prior with multiple historical controls for the linear regression model

“…The selection of historical data should be based on careful consideration, guided by Pocock's rules 9 or a reasonable extension thereof as in Hatswell et al. 17 The design of the simulation study was based on an assumption of exchangeability of the historical studies with the current study. Therefore the simulation results do not apply to any type of prior-data conflict where the outcomes of the current study have a systematic bias compared to the historical studies, for example, when there are secular improvements in health outcomes over time.…”