A meso-level empirical validation approach for agent-based computational economic models drawing on micro-data: a use case with a mobility mode-choice model

Abstract

The complex nature of agent-based modeling may reveal more descriptive accuracy than analytical tractability. That leads to an additional layer of methodological issues regarding empirical validation, which is an ongoing challenge. This paper offers a replicable method to empirically validate agent-based models, a specific indicator of “goodness-of-validation” and its statistical distribution, leading to a statistical test in some way comparable to the p value. The method involves an unsupervised machine learning algorithm hinging on cluster analysis. It clusters the ex-post behavior of real and artificial individuals to create meso-level behavioral patterns. By comparing the balanced composition of real and artificial agents among clusters, it produces a validation score in [0, 1] which can be judged thanks to its statistical distribution. In synthesis, it is argued that an agent-based model can be initialized at the micro-level, calibrated at the macro-level, and validated at the meso-level with the same data set. As a case study, we build and use a mobility mode-choice model by configuring an agent-based simulation platform called BedDeM. We cluster the choice behavior of real and artificial individuals with the same ex-ante given characteristics. We analyze these clusters’ similarity to understand whether the model-generated data contain observationally equivalent behavioral patterns as the real data. The model is validated with a specific score of 0.27, which is better than about 95% of all possible scores that the indicator can produce. By drawing lessons from this example, we provide advice for researchers to validate their models if they have access to micro-data.

How, where, and when to charge electric vehicles – net-zero energy system implications and policy recommendations

“…Thus, the CI options represent countrywide charging nodes but are not allocated to GIS-based locations. Agent-based transport simulation models with a finer geographical resolution [109][110][111][112] and dedicated EV charging models [113] could be better suited for a detailed assessment of the ideal type and number of charging points needed at certain locations. However, such models typically lack in assessing cross-sectoral impacts and have other disadvantages [114].…”

Towards more impactful energy research: The salient role of social sciences and humanities

“…Technological advances in information acquisition, such as mobile sensing approaches, enable more precise measurements as complementary to existing SSH approaches. Moreover, human-centered modeling approaches such as agentbased modeling can transfer SSH findings on individual decision-making and behavior from the individual level to the level of larger social systems [13]. These approaches further enable the analysis of dynamic behavior and social interactions across time.…”

A Behavioural Decision-Making Framework For Agent-Based Models

“…As our models are built on a behavioural theory framework, we need to validate our agent-based models empirically. Bektas et al proposed to use an unsupervised machine learning algorithm based on cluster analysis of real and artificial individuals to create meso-level behavioural patterns [BPS21]. The algorithm generates a validation score by comparing the balanced composition of real and artificial agents among these clusters.…”

“…The algorithm generates a validation score by comparing the balanced composition of real and artificial agents among these clusters. This method was applied in the mobility case studies [BPS21]. It can also be done similarly for the vehicle purchasing study.…”