Validation and verification of social processes within agent-based computational organization models

Abstract

The use of simulation modeling in computational analysis of organizations is becoming a prominent approach in social science research. However, relying on simulations to gain intuition about social phenomena has significant implications. While simulations may give rise to interesting macro-level phenomena, and sometimes even mimic empirical data, the underlying micro and macro level processes may be far from realistic. Yet, this realism may be important to infer results that are relevant to existing theories of social systems and to policy making. Therefore, it is important to assess not only predictive capability but also explanation accuracy of formal models in terms of the degree of realism reflected by the embedded processes. This paper presents a process-centric perspective for the validation and verification (V&V) of agent-based computational organization models. Following an overview of the role of V&V within the life cycle of a simulation study, emergent issues in agent-based organization model V&V are outlined. The notion of social contract that facilitates capturing micro level processes among agents is introduced to enable reasoning about the integrity and consistency of agent-based organization designs. Social contracts are shown to enable modular compositional verification of interaction dynamics among peer agents. Two types of consistency are introduced: horizontal and vertical consistency. It is argued that such local consistency analysis is necessary, but insufficient to validate emergent macro processes within multi-agent organizations. As such, new formal validation metrics are introduced to substantiate the operational validity of emergent macro-level behavior.

Lessons for model use in transition research: A survey and comparison with other research areas

“…If the chosen micro-level structure is able to generate the observed macro phenomena, the model provides a potential explanation for these phenomena. Models should base upon a sound theoretical foundation and use empirical data and expert knowledge to substantiate the model's micro-foundation (Boero and Squazzoni, 2005;Windrum et al, 2007;Yilmaz, 2006).…”

Adaptation and Validation of an Agent Model of Functional State and Performance for Individuals

“…In particular, a validation approach from (Yilmaz, 2006) distinguishes the validation phases similar to the ones considered in the paper (e.g., conceptual and operational validation); however, the precise elaboration of the phases is focused largely on social processes, which are not relevant for our work. Furthermore, examples of model validation are found in psychology, e.g.…”

Agent-based simulation in management and organizational studies: a survey

“…ABS can have programming errors or might not adequately capture the essence of the target system. There are no standardized models that guarantee verification and validation in ABS, in spite of recent efforts (Yilmaz, 2006). Also, interpreting the results of an ABS is hard when they are counterintuitive or when there is stochasticity involved.…”