Selecting a Dynamic Simulation Modeling Method for Health Care Delivery Research—Part 2: Report of the ISPOR Dynamic Simulation Modeling Emerging Good Practices Task Force

Abstract

In a previous report, the ISPOR Task Force on Dynamic Simulation Modeling Applications in Health Care Delivery Research Emerging Good Practices introduced the fundamentals of dynamic simulation modeling and identified the types of health care delivery problems for which dynamic simulation modeling can be used more effectively than other modeling methods. The hierarchical relationship between the health care delivery system, providers, patients, and other stakeholders exhibits a level of complexity that ought to be captured using dynamic simulation modeling methods. As a tool to help researchers decide whether dynamic simulation modeling is an appropriate method for modeling the effects of an intervention on a health care system, we presented the System, Interactions, Multilevel, Understanding, Loops, Agents, Time, Emergence (SIMULATE) checklist consisting of eight elements. This report builds on the previous work, systematically comparing each of the three most commonly used dynamic simulation modeling methods-system dynamics, discrete-event simulation, and agent-based modeling. We review criteria for selecting the most suitable method depending on 1) the purpose-type of problem and research questions being investigated, 2) the object-scope of the model, and 3) the method to model the object to achieve the purpose. Finally, we provide guidance for emerging good practices for dynamic simulation modeling in the health sector, covering all aspects, from the engagement of decision makers in the model design through model maintenance and upkeep. We conclude by providing some recommendations about the application of these methods to add value to informed decision making, with an emphasis on stakeholder engagement, starting with the problem definition. Finally, we identify areas in which further methodological development will likely occur given the growing "volume, velocity and variety" and availability of "big data" to provide empirical evidence and techniques such as machine learning for parameter estimation in dynamic simulation models. Upon reviewing this report in addition to using the SIMULATE checklist, the readers should be able to identify whether dynamic simulation modeling methods are appropriate to address the problem at hand and to recognize the differences of these methods from those of other, more traditional modeling approaches such as Markov models and decision trees. This report provides an overview of these modeling methods and examples of health care system problems in which such methods have been useful. The primary aim of the report was to aid decisions as to whether these simulation methods are appropriate to address specific health systems problems. The report directs readers to other resources for further education on these individual modeling methods for system interventions in the emerging field of health care delivery science and implementation.

Una comparación cualitativa de la dinámica de sistemas, la simulación de eventos discretos y la simulación basada en agentes

“…La tabla 1 contiene una recopilación de los elementos distintivos de cada paradigma, expuestos por diversos autores (Marshall et al, 2015b;Behdani, 2012;Tacko y Robinson, 2009 …”

NETIMIS: Dynamic Simulation of Health Economics Outcomes Using Big Data

“…Following this checklist, our problem required modelling multiple events as processes (System), including non-linear relationships that make predicting outcomes difficult (Interactions), modelling systems at different levels (Multi-level), modeling complexity (Understanding), modeling feedback (Loops), interactions between entities (Agents), time-dependent behavior (Time) and surfacing emergent behavior (Emergence). The available tools can be broadly classified as based on systems dynamics, discrete event simulation or agent-based modelling though the Task Force note the increasing popularity of emerging hybrid models [18]. NETIMIS (Network Tools for Intervention Modelling with Intelligent Simulation) is classed as a discrete event simulation tool that includes some hybrid elements.…”

“…The team included health economists, software engineers, data miners, business analysts and clinicians from the study areas. The approach combined dynamic simulation modelling good practice [18], data mining methods (including statistical analysis, extract transform load, data cleansing), with modelling methods (care pathways [40], process modelling, economic modelling), primary data collection (including observation and interviews) [37] and data quality methods [16] (notably comparison between sources, validity checks and "gold standard" clinical review). The end of each iteration was marked by a team meeting to review the model with the focus moving from plausibility, to completeness to correctness with each session driving questions and suggested methods for the next iteration of investigation.…”

“…The NETIMIS tool can be used for hypothesis generation and exploratory research to support studies that evaluate healthcare interventions. Two recent publications from the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Task Force on Simulation Modeling Applications in Health Care Delivery Research [17,18] make a case that standardized methods for evaluating healthcare interventions (including decision trees and Markov modeling) are not sufficient for analyzing healthcare delivery systems. They advocate the use of dynamic simulation tools to help better understand the complex relationships in healthcare systems.…”

Discrete Event Simulation-Based Resource Modelling in Health Technology Assessment

“…patients) moving through the system, and the changes in the states of the entities at discrete time points [3,4]. Importantly, DES provides the capability to model resource constraints explicitly [5,6,7].…”