Measuring and Evaluating Parallel State-Space Exploration Algorithms

Abstract

We argue in this paper that benchmarking should be complemented by direct measurement of parallelisation overheads when evaluating parallel state-space exploration algorithms. This poses several challenges that so far have not been addressed in the literature: what exactly are those overheads, how can and cannot they be measured, and how should system models be selected in order to expose the causes of parallelisation (in)efficiencies? We discuss and answer these questions based on our experience with parallelising Saturation -a symbolic algorithm for generating state-spaces of asynchronous system models -on a shared-memory architecture. Doing so will hopefully spare newcomers to the growing PDMC community from having to learn these lessons the hard way, as we did over a painful period of almost three years.

Modelling parallel overhead from simple run-time records

“…In general, we consider every incremental time fragment emerging within a parallel algorithm parallel overhead if it is in excess of the serial algorithm required to solve exactly the same type of problem. Typically this will include [15], -time to interchange data -time to synchronize individual parallel tasks -extra computing time due to code sections arising only in the parallel algorithm -computing time penalties due to load balancing issues -computing time penalties due to inhomogeneous conditions between individual components of the parallel machine [1] Measuring parallel overhead is not a trivial matter [29][30][31]. A conventional view is that to a large extent it is all covered by communication overhead.…”

Mixed Shared-Distributed Hash Tables Approaches for Parallel State Space Construction

“…For instance, our efficiency 1 may vary between 90% (Hanoi model) and 51% (Kanban model), whereas the system occupancy rate 2 is consistently over 95%. Clearly, the algorithm depends on the "degree of concurrency" of the model -it is not necessary to use lots of processors for a model with few concurrent actions -but this is an inherent limitation with parallel state space construction [3], which is an irregular problem. Concerning the use of memory, we can measure the quality of the distribution of the state space using the mean standard deviation (σ) of the number of states among the processors.…”

“…This technique relies on the exploration of the state space of the model. State space construction can be classified as an irregular parallel problem because state graphs may be highly irregular, see [3] for a discussion on this topic. As a consequence, when parallelizing this problem, special attention should be taken to ensure a good load balancing among processors.…”

Using Model Checking Techniques for Symbolic Synthesis of Distributed Programs

“…Parallelization of symbolic reachability analysis has been studied in the model checking community from different perspectives. In [9][10][11], the authors propose solutions and analyze different approaches to parallelization of the saturation-based generation of state space in model checking.…”