A Formula-Driven Scalable Benchmark Model for ABM, Applied to FLAME GPU

Abstract

Agent Based Modelling (ABM) systems have become a popular technique for describing complex and dynamic systems. ABM is the simulation of intelligent agents and how these agents communicate with each other within the model. The growing number of agent-based applications in the simulation and AI fields led to an increase in the number of studies that focused on evaluating modelling capabilities of these applications. Observing system performance and how applications behave during increases in population size is the main factor for benchmarking in most of these studies. System scalability is not the only issue that may affect the overall performance, but there are some issues that need to be dealt with to create a standard benchmark model that meets all ABM criteria. This paper presents a new benchmark model and benchmarks the performance characteristics of the FLAME GPU simulator as an example of a parallel framework for ABM. The aim of this model is to provide parameters to easily measure the following elements: system scalability, system homogeneity, and the ability to handle increases in the level of agent communications and model complexity. Results show that FLAME GPU demonstrates near linear scalability when increasing population size and when reducing homogeneity. The benchmark also shows a negative correlation between increasing the communication complexity between agents and execution time. The results create a baseline for improving the performance of FLAME GPU and allow the simulator to be contrasted with other multi-agent simulators.

Data Aware Simulation of Complex Systems on GPUs

“…To evaluate the benefits of using the proposed system, this section will show the comparison of results between the current FLAME GPU and the modified version using the benchmark model that was proposed by [31]. The benchmark model is based on the concept of particle-based simulation and accepts input parameters that control both system scalability and agent homogeneity.…”

“…The population size of each agent type starts with 100,000 agents and ends with 800,000 agents. This benchmark is based on the same example that was used by Alzahrani et al [31]. It is representative of scaling the population size of the model and the simulation was performed for 100 iterations.…”

“…The main concept of this experiment is to observe the effects of divergent behaviour (within an agent) on the execution time for both systems. Based on the same example that is used by the benchmark model within [31], this benchmark is representative of increasing the individual complexity of an agent, and that means more functions in each layer every cycle. As the function layers represent the control flow of simulation processes in FLAME GPU, adding more agent functions every time will increase the number of layers in each cycle (as functions of the same agent need to be processed in sequential order) and that will lead to increasing the execution time of each iteration.…”

Designing a benchmark for the performance evaluation of agent-based simulation applications on HPC

“…To the best of our knowledge, the most serious attempts to create a benchmark for assessing parallel ABMS applications performance comes from the OpenAB group [13] and were published in [30,31].…”