Replicating agent-based models: Revisiting March’s exploration–exploitation study

Abstract

To validate prior agent-based models, public disclosure of model code is necessary, but not sufficient. Conceptual model replication, involving independent reproduction of a model without referring to the originator’s code, is crucial for detecting nonconformity between the publication text and the model implemented. We frame and evaluate four scenarios to expose the shortfalls of replication efforts that use the original study’s program code and neglect the published model description. We identify, evaluate, and offer recommendations regarding disparities between the text and the program code that generated the results reported in March’s classic study of exploration and exploitation in organizational learning. Based on our experience replicating his model, we suggest a two-step method of model verification—beginning with replicating the model from the published description, then turning to the program code of the original implemented model to account for divergent results. Exclusive reliance on either the published model description or the original program code does disservice to theory advancement, because disparities between the original conceptual and implemented models go unaddressed.

Impacts of Practice Combinations on Organizational Knowledge: Based on March’s Exploration-Exploitation Model

“…Comparing results in Figure 2 with those in Chanda and Miller's [33] Figure 1, all thirteen curves generate consistent, quantitative results, when allowing for small variation produced by random processes across runs of the model. e result of the comparison directly proves the correctness of our model.…”

“…For each model variant, Figures 2(a Comparing the corresponding curve, including trend and numerical value with March's original results, Figure 1 shows qualitative similarities, which enable us to draw the main conclusions of his paper, but always with quantitative differences, especially in Figure 2(e). Chanda and Miller [33] find the same phenomenon in their paper. ey identify, compare, and evaluate March's descriptions of the model and his original program code systematically.…”

“…Most relevant existing literature does not involve quantitative comparison with March's original model, just qualitative comparison, even no. According to Chanda and Miller [33], this operation does not guarantee the reconstructed model complete consistency with the original model, which will In conclusion, they propose that the results presented in Figure 1 (pp. 10) should be considered as the baseline for future replication and extension work based on March's model.…”

When Is More Uncertainty Better? A Model of Uncertainty Regulation and Effectiveness

“…We suggest that computational modelling is well-suited to examine these intertwined processes of uncertainty regulation. This approach was implemented in the original research by March (1991) on exploration and exploitation in organizations and recently used to replicate his results (Chanda & Miller, 2018).…”

Organizational Adaptability Influenced by Practice Strategy, Environmental Dynamism, and Absorptive Capacity

“…Although this does not affect the validity of the main conclusions in March's article, it would be a destructive blow to the robustness of our findings, as the correct establishment of the model is the cornerstone for further extensions. After a targeted search, we find that Chanda and Miller [39] encountered the same problem in their study. e difference is that they managed to get the source code from Professor James March.…”