Computer Modeling of Diabetes and Its Transparency: A Report on the Eighth Mount Hood Challenge

Abstract

Objectives: The Eighth Mount Hood Challenge (held in St. Gallen, Switzerland, in September 2016) evaluated the transparency of model input documentation from two published health economics studies and developed guidelines for improving transparency in the reporting of input data underlying model-based economic analyses in diabetes. Methods: Participating modeling groups were asked to reproduce the results of two published studies using the input data described in those articles. Gaps in input data were filled with assumptions reported by the modeling groups. Goodness of fit between the results reported in the target studies and the groups' replicated outputs was evaluated using the slope of linear regression line and the coefficient of determination (R 2 ). After a general discussion of the results, a diabetes-specific checklist for the transparency of model input was developed. Results: Seven groups participated in the transparency challenge. The reporting of key model input parameters in the two studies, including the baseline characteristics of simulated patients, treatment effect and treatment intensification threshold assumptions, treatment effect evolution, prediction of complications and costs data, was inadequately transparent (and often missing altogether). Not surprisingly, goodness of fit was better for the study that reported its input data with more transparency. To improve the transparency in diabetes modeling, the Diabetes Modeling Input Checklist listing the minimal input data required for reproducibility in most diabetes modeling applications was developed. Conclusions: Transparency of diabetes model inputs is important to the reproducibility and credibility of simulation results. In the Eighth Mount Hood Challenge, the Diabetes Modeling Input Checklist was developed with the goal of improving the transparency of input data reporting and reproducibility of diabetes simulation model results.

Systematic Literature Review and Critical Appraisal of Health Economic Models Used in Cost-Effectiveness Analyses in Non-Alcoholic Steatohepatitis: Potential for Improvements

“…For example, the Mount Hood Diabetes Challenge Network is an informal association of diabetes modellers that holds regular meetings with an aim of promoting transparency though validation and allows for internal, external and cross-validation of available diabetes models [56]. Previous meetings have involved transparency challenges wherein model developers are tasked with reproducing the results of clinical and cost-effectiveness analyses, with differences between models compared and rationale sought for any major differences [56,58]. The Mount Hood group have also developed the Diabetes Modelling Input Checklist, a disease-specific checklist [58].…”

A Systematic Review of Model-Based Economic Evaluations of Treatments for Venous Leg Ulcers

“…Since none of these studies published the methodology or results of their reviews separately, this may have resulted in a duplication of researcher effort, which could have been focused on other areas of the modelling process, such as model validation. There is also potential for future research to develop guidelines specifically for VLU-based models, similar to guidelines developed by the Mt Hood Diabetes Challenge Network [60]. A similar review focusing on trialbased economic evaluations for VLU treatments would also be of worth, as there is still a significant gap in the literature for this area.…”

Development and Validation of PREDICT-DM: A New Microsimulation Model to Project and Evaluate Complications and Treatments of Type 2 Diabetes Mellitus

“…There are a number of existing simulation models of T2DM. 6 These models have been very helpful in informing policy questions regarding diabetes care in the United States and worldwide. [7][8][9] Given substantial advances in T2DM care over the past two decades, 10 these models have been updated with new equations and calibration methods.…”