Strengthening the reporting of empirical simulation studies: Introducing the STRESS guidelines

Abstract

This study develops a standardised checklist approach to improve the reporting of discrete-event simulation, system dynamics and agent-based simulation models within the field of Operational Research and Management Science. Incomplete or ambiguous reporting means that many simulation studies are not reproducible, leaving other modellers with an incomplete picture of what has been done and unable to judge the reliability of the results. Crucially, unclear reporting makes it difficult to reproduce or reuse findings. In this paper, we review the evidence on the quality of model reporting and consolidate previous work. We derive general good practice principles and three 20-item checklists aimed at Strengthening The Reporting of Empirical Simulation Studies (STRESS): STRESS-DES, STRESS-ABS and STRESS-SD for discrete-event simulation, agentbased simulation and system dynamics, respectively. Given the variety of simulation projects, we provide usage and troubleshooting advice to cover a wide range of situations.

How simulation modelling can help reduce the impact of COVID-19

“…Appropriate engagement will require a good communication of the problem situation and that could be done with the support of a problem structuring method (Mingers, 2001) but there is still a need to translate them for a virtual environment and a need for training modellers in soft skills. The modelling and simulation community has generally approached modelling in the expert mode (favouring observation and data driven models) rather than the facilitated mode of interaction (favouring expert views in addition to data), which would necessitate some collaboration with facilitators and problem structurers or upskilling the modelling and simulation community.…”

Towards generic modelling of hospital wards: Reuse and redevelopment of simple models

“…The second study tested the generic properties of the first model in a more specialised (intensive care) ward and identified additional flexibility needed in order to successfully reuse it. As both studies were informing real decision-making, our work also contributes to the growing, but still limited, evidence about the impact of computer simulation in health (Brailsford & Vissers, 2011;Fone et al, 2003;Günal & Pidd, 2010;Monks et al, 2018;Crowe, Turner, Utley, & Fulop, 2017). In each case, we describe the use of the models and their results in practice and the challenges faced in reuse.…”

“…The coded models are implemented in Simul8 Professional 2018. We document the model using the STRESS-DES reporting guidelines (Monks et al, 2018). The full STRESS-DES documentation is available in the online appendix.…”

“…As the creation and publication of generic models increases it would be beneficial to develop a metric to define the level of generality of models in a standardised manner. This might also include standardisation of documentation of models in a similar way to the STRESS-DES guidelines (Monks et al, 2018), specifically focussed on the scope of the model to adapt to different system features and performance measures.…”

Capacity and patient flow planning in post-term pregnancy outpatient clinics: a computer simulation modelling study

“…Jun et al developed a tool to guide in the selection of the most appropriate Operational Research (OR) method(s), including simulation, to support health services management [19]. We follow the STRESS guidelines to document the model presented [20].…”