The right algorithm at the right time: comparing data flow analysis algorithms for finite state verification

Abstract

Finite state verification is emerging as an important technology for proving properties about software. In our experience, we have found that analysts have different expectations at different times. When an analyst is in an exploratory mode, initially formulating and verifying properties, analyses usually find inconsistencies because of flaws in the properties or in the software artifacts being analyzed. Once an inconsistency is found, the analyst begins to operate in a fault finding mode, during which meaningful counter example traces are needed to help determine the cause of the inconsistency. Eventually systems become relatively stable, but still require re-verification as evolution occurs. During such periods, the analyst is operating in a maintenance mode and would expect re-verification to usually report consistent results. Although it could be that one algorithm suits all three of these modes of use, the hypothesis explored here is that each would be best served by an algorithm optimized for the expectations of the analyst.

Flow analysis for verifying properties of concurrent software systems

“…We have studied how different variations of the state-propagation algorithm can impact performance, depending on whether consistent or inconsistent results are expected or if short counterexample traces are desired [Cobleigh et al 2001]. Path length is only one of the important attributes of counterexamples, however.…”

“…For inconclusive results, there are usually several counterexamples that could be shown, and the order that these are generated in depends on how the worklist is implemented in the state-propagation algorithm [Cobleigh et al 2001]. For instance, FLAVERS might return the path 20, 1, 9, 21, 2, 11, 22, 4, 14b, 23, 2, 18, 11, 22, 4, 14b, 23, 2, 18, 11, 22, 4, 14a, 26, 8, 19, 27.…”

Heuristic-based model refinement for FLAVERS

“…The comparison of performance is significantly trickier in the case where the properties do not hold, and previous research [5] has shown that different algorithms may be appropriate if FLAVERS is applied early in development when many bugs remain and the properties being checked are not likely to hold in most cases. We intend to investigate the performance of these heuristics in such cases and to look for other heuristics that may give better performance.…”

“…), evaluating this quality is complex. Moreover, as shown in [5], it may be advantageous to modify the FLAVERS algorithm in cases where inconclusive results are expected, and then additional heuristics to guide the search for counterexamples are likely to be helpful [12]. In future work, we intend to examine the use of heuristics when properties are violated.…”

“…The goals have been to find shorter paths or to improve performance, or both [5,12,14,28]. The heuristics have been based on the property or on the model, as with our heuristics, as well as on violationstate information.…”