Validation of Computer Simulations from a Kuhnian Perspective

Abstract

While Thomas Kuhn's theory of scientific revolutions does not specifically deal with validation, the validation of simulations can be related in various ways to Kuhn's theory: 1) Computer simulations are sometimes depicted as located between experiments and theoretical reasoning, thus potentially blurring the line between theory and empirical research. Does this require a new kind of research logic that is different from the classical paradigm which clearly distinguishes between theory and empirical observation? I argue that this is not the case. 2) Another typical feature of computer simulations is their being "motley" (Winsberg, 2003) with respect to the various premises that enter into simulations. A possible consequence is that in case of failure it can become difficult to tell which of the premises is to blame. Could this issue be understood as fostering Kuhn's mild relativism with respect to theory choice? I argue that there is no need to worry about relativism with respect to computer simulations, in particular.3) The field of social simulations, in particular, still lacks a common understanding concerning the requirements of empirical validation of simulations. Does this mean that social simulations are still in a pre-scientific state in the sense of Kuhn? My conclusion is that despite ongoing efforts to promote quality standards in this field, lack of proper validation is still a problem of many published simulation studies and that, at least large parts of social simulations must be considered as pre-scientific.

The computational philosophy: simulation as a core philosophical method

“…Excepting the Munich Center for Mathematical Philosophy, we are aware of no graduate programs in philosophy that require a modeling or programming course. Some philosophers have recently taken aim at the value of the agent-based models most common in philosophy (Arnold 2014 , 2015 , 2019 ; Thicke 2019 ). Finally, as modelers, we can attest to hearing the following complaint time and again: “Interesting, but why is your research philosophical ?” 3…”

The Sociotechnical Digital Twin: On the Gap Between Social and Technical Feasibility

“…First, in the initial design of the sociotechnical DT, by presenting the primary objective of the DT as a thoughtexperiment, empirical validation is not always required. Second, and this is related to the abstraction gap, discussed earlier, by focusing on a tightly restricted topic, it is possible to find the right level of abstraction [3]. Such an example is illustrated in the area of organisational decision-making [6].…”

“…Epistemic concerns that arise directly out of simulation as a form of scientific experiment are prevalent [34]. Epistemic concerns are further compounded by the acceptance that sociotechnical DT will invariably incorporate theories from the social sciences which are multi-paradigm sciences where validations generate specific issues [3] such that validation is rarely present. Social simulations may draw upon multiple theories, approximations and data sets such that when a validation if it exists, fails, the complexity makes it challenging to identify the point of failure.…”

The Strategic Advantage of Impulsivity in Entrepreneurial Action: An Agent-Based Modeling Approach

“…Thus, in terms of establishing validity, the modeling assumptions are calibrated based on the previously established findings (cf. Arnold, 2019). Consistent with best practices, we use open-source ABM software, NetLogo (version 6.1), a tool often used by entrepreneurship scholars (e.g., Bylund, 2015) and the broader scientific community (Wilensky & Rand, 2015).…”