Noise-based testing and analysis of multi-threaded C/C++ programs on the binary level

Advances in noise‐based testing of concurrent software

“…Both of these heuristics target common atomicity violation scenarios, and the newly proposed noise seeding heuristic might also help in order violation scenarios. The newly proposed heuristics are compared with a selection of already existing heuristics, which provided promising results in the previous experimental comparisons . The presented set of eight Java benchmark programmes and four C benchmark programmes makes the comparison the so‐far largest comparison of noise‐injection‐based testing techniques.…”

“…Additionally, several new heuristics are introduced. Then, in Section 4.3, a selection of results of previously published comparisons of the older noise‐injection heuristics is presented. Based on the results, the most promising heuristics and their parameters are pinpointed.…”

“…Next, a noise placement heuristic called read/write heuristic uses different noise settings for the shared variable read accesses and write accesses. The settings might differ in the frequency that controls how often a noise is generated before a particular class of accesses or in the chosen noise seeding heuristic.…”

“…In this paper, a unified overview of multiple results from the area of noise‐injection‐based testing that were published by the authors in several past years is provided. In particular, multiple novel coverage metrics (first proposed by Křena et al ), which measure how well the behaviour of tested programmes has been examined from the point of view of possible occurrence of certain synchronization‐related errors, are presented first.…”

“…These metrics can be used to control saturation‐based testing, to compare effectiveness of various testing approaches, or to tune parameters of meta‐heuristic algorithms applied in testing . Next, various noise‐injection heuristics and their influence on error detection and ability to increase concurrency‐related coverage (absolutely and relatively, i.e. taking into account the longer test execution time caused by noise injection) are studied.…”