SUMMARYTesting is a key activity to assure the quality of concurrent applications. In recent years, a variety of different mechanisms have been proposed to test concurrent software. However, a persistent problem is the high testing cost because of the large number of different synchronization sequences that must be tested. When structural testing criteria are adopted, a large number of infeasible synchronization sequences is generated, increasing the testing cost. Although the use of reachability testing reduces the number of infeasible combination (because only feasible synchronization sequences are generated), many synchronization combinations are also generated, and this again results in a testing cost with exponential behavior. This paper presents a new composite approach that uses reachability testing to guide the selection of the synchronization sequences tests according to a specific structural testing criterion. This new composite approach is empirically evaluated in the context of message-passing concurrent programs developed with MPI. The experimental study evaluates both the cost and effectiveness of proposed composite approach in comparison with traditional reachability testing and structural testing. The results confirm that the use of the new composite approach has advantages for testing of concurrent applications. Copyright © 2015 John Wiley & Sons, Ltd. KEY WORDS: structural testing; reachability testing; concurrent programs; experimental study
INTRODUCTIONConcurrent software is being increasingly used with the availability of multicore processors and computer clusters. Many modern business applications now use concurrency to improve overall system performance. However, all concurrent software contains features, such as nondeterminism, synchronization, and interprocess communication, which make concurrent software significantly more complex than sequential software. These features impose new challenges for testing and increase the difficulty of demonstrating the correct execution of the application in all possible conditions. Traditional testing techniques are often not well-suited to the testing of concurrent (or parallel) software. Many researchers have developed specific testing techniques to address specific issues such as nondeterminism, concurrency, communication, synchronization, race conditions, and replay testing [1][2][3][4][5][6][7][8][9][10][11].In previous work, structural testing criteria for concurrent programs were proposed by Souza et al. [12], defining test models and tools to test message-passing software. This work was extended by Sarmanho et al. [13] to test shared-memory software. These structural testing criteria are designed to explore information about the control, data, and communication flows of concurrent programs, considering both their sequential and parallel aspects. These contributions are related to the coverage analysis concept, in which a coverage model is first defined, and then the model is evaluated during the execution of the program to determine whether th...