The Use of Computer Simulation Modeling to Estimate Complications in Patients with Type 2 Diabetes Mellitus: Comparative Validation of the Cornerstone Diabetes Simulation Model

Abstract

Objective The objective of this study was to assess the validity of the Cornerstone Diabetes Simulation (CDS), a Microsoft Excel ®-based patient-level simulation for type 2 diabetes mellitus based on risk equations from the revised United Kingdom Prospective Diabetes Study Outcomes Model (UKPDS-OM2, also known as UKPDS 82). Methods Three levels of validation were conducted. Internal validation was assessed through independent review and model stress-testing. External validation was addressed by populating the CDS with baseline characteristics and treatment effects from three major diabetes clinical trials used in the Fifth Mount Hood Diabetes Challenge (MH5) for computer simulation models. Cross-validation of predicted outcomes was tested versus eight models that participated in the MH5. Simulated results were compared with observed clinical outcomes via the coefficient of determination (R 2) for both the absolute risk of each clinical outcome and the difference in absolute risk between control and intervention arm in each trial. We ensured transparency of all model inputs and assumptions in reporting. Results The CDS could be used to predict 18 of 39 single and composite endpoints across the three trials. The model obtained an R 2 of 0.637 for predicted versus observed absolute risks, and an R 2 of 0.442 for predicted versus observed risk differences between control and intervention. Among the other eight models, only one obtained a higher R 2 value under both definitions, albeit based on only four predicted endpoints. Conclusions The CDS provides good predictions of diabetes-related complications when compared to observed trial outcomes and previously validated models. The model has value as a validated tool in cost-effectiveness evaluations.

Decision models in type 2 diabetes mellitus: A systematic review

“…Eleven models [10,11,[20][21][22][23][24][25][26][27][28] simulated annual changes in risk factors such as body mass index (BMI), glycemia, HbA1c, blood pressure (systolic and/or diastolic), and lipids (total cholesterol and/or high-density lipoprotein) (Table 2). The simulated trajectory of risk factors could affect the subsequent occurrence or development of diabetes and its complications.…”

“…Twelve models [11,12,[19][20][21][22][23][24][25][26][27][28] reported life-years (LYs), ten model [11,12,19,20,[22][23][24][25][26][27] reported incremental costeffectiveness ratios (ICERs), and thirteen models [10][11][12][19][20][21][22][23][24][25][26][27][28] reported quality-adjusted life years (QALYs). The ECHO and IHE models also reported net monetary benefits (NMBs).…”

“…All models reported costs, albeit at different levels of detail. Eleven models [9,11,12,19,20,[22][23][24][25][26][27] reported direct costs, whereas the CDM and IHE models reported both direct and indirect costs. Three models (UKPDS OM1/2 and the Michigan model) did not describe cost in detail.…”

“…Most CUA were made by calculating QALYs. Some models [11,12,19,20,[22][23][24][25][26][27] also took ICERs into account and thus could perform incremental analyses.…”

“…Eleven of fourteen primary studies reported that one or more validation checks had been performed. Four studies [10,24,26,28] presented model face validation, eleven studies [9,10,[19][20][21][23][24][25][26][27][28] presented internal validation, ten studies [10,[19][20][21][23][24][25][26][27][28] presented external validation, while cross-validation was conducted by three studies [24,25,28]. However, none of the 14 studies demonstrated predictive validation.…”