A methodological framework for assessing agreement between cost-effectiveness outcomes estimated using alternative sources of data on treatment costs and effects for trial-based economic evaluations

Abstract

A new methodological framework for assessing agreement between cost-effectiveness endpoints generated using alternative sources of data on treatment costs and effects for trial-based economic evaluations is proposed. The framework can be used to validate cost-effectiveness endpoints generated from routine data sources when comparable data is available directly from trial case report forms or from another source. We illustrate application of the framework using data from a recent trial-based economic evaluation of the probiotic Bifidobacterium breve strain BBG administered to babies less than 31 weeks of gestation. Cost-effectiveness endpoints are compared using two sources of information; trial case report forms and data extracted from the National Neonatal Research Database (NNRD), a clinical database created through collaborative efforts of UK neonatal services. Focusing on mean incremental net benefits at £30,000 per episode of sepsis averted, the study revealed no evidence of discrepancy between the data sources (two-sided p values >0.4), low probability estimates of miscoverage (ranging from 0.039 to 0.060) and concordance correlation coefficients greater than 0.86. We conclude that the NNRD could potentially serve as a reliable source of data for future trial-based economic evaluations of neonatal interventions. We also discuss the potential implications of increasing opportunity to utilize routinely available data for the conduct of trial-based economic evaluations.

Information technology infrastructure, quality improvement and research: the UK National Neonatal Research Database

“…The NNRD is also widely used for research studies. These include health services research (13)(14)(15)(16)(17), health economics (18) and epidemiological studies (19)(20)(21). The NNRD is used to provide clinical decision support analyses, for example in relation to gestational age specific mortality (22), length of stay (23), development of a gestational case definition for necrotising enterocolitis (24), care of preterm babies with Downs Syndrome (25) and comparative evaluation of preterm growth (26).…”

Assessing the Health Economic Agreement of Different Data Sources

“…The PMC is recommended by Achana et al (2018) to be implemented by taking the INB from the routine data source and the confidence intervals of the gold-standard trial data. This will then yield the probability that the confidence intervals do not contain the INB estimate, allowing for assessment of the agreement between the sets of data.…”

“…is the reference INB to be used in the PMC. Achana et al (2018) suggest using the INB from one dataset as the reference and comparing it with the 95% confidence intervals of the other dataset.…”

“…Achana et al (5) introduced a methodological framework for the assessment of agreement, utilising Lin's Concordance Correlation Coefficient (CCC)(6), the difference in Incremental Net Benefit (INB) estimates, the Probability of Miscoverage (PMC) and the Probability of Cost Effectiveness (PCE). Building on this work, generalisable commands are presented allowing these analyses to be performed with relative ease.…”

“…For example, they can be used to compare and validate routine data for use in trial-based economic evaluations when alternative sources of information on costs and effects are available for trial participants. Achana et al (2018) introduced a methodological framework for the assessment of agreement using Lin's (1989) concordance correlation coefficient (CCC), the difference in incremental net benefit (INB) estimates, the probability of miscoverage (PMC), and the probability of cost effectiveness (PCE). Building on this work, we present generalizable commands allowing these analyses to be performed with relative ease.…”