Guided test generation for database applications via synthesized database interactions

Abstract

Testing database applications typically requires the generation of tests consisting of both program inputs and database states. Recently, a testing technique called Dynamic Symbolic Execution (DSE) has been proposed to reduce manual effort in test generation for software applications. However, applying DSE to generate tests for database applications faces various technical challenges. For example, the database application under test needs to physically connect to the associated database, which may not be available for various reasons. The program inputs whose values are used to form the executed queries are not treated symbolically, posing difficulties for generating valid database states or appropriate database states for achieving high coverage of query-result-manipulation code. To address these challenges, in this article, we propose an approach called SynDB that synthesizes new database interactions to replace the original ones from the database application under test. In this way, we bridge various constraints within a database application: query-construction constraints, query constraints, database schema constraints, and query-result-manipulation constraints. We then apply a state-of-the-art DSE engine called Pex for .NET from Microsoft Research to generate both program inputs and database states. The evaluation results show that tests generated by our approach can achieve higher code coverage than existing test generation approaches for database applications.

Relational symbolic execution of SQL code for unit testing of database programs

“…Our last objective with the experiments was to compare the performance of relational symbolic execution with the SynDB tool proposed in [23]. In this tool, the tested code is preprocessed to translate the SQL code into normalised native program code, before applying a classical symbolic execution process.…”

“…In order to propose a fair comparison, the total generation time for each method was measured on a Pentium 4 configuration (3.06GHz, 1GB RAM, Microsoft Windows XP 32-bit), similar (but admittedly not identical) to the one used in [23]. The obtained results are synthesised in the first part of table 8.…”

“…A first reason for such a performance difference between both tools is that [23] normalises the SQL code into native code before trying to generate tests. This requires time and significantly increases the number of paths to be explored, as each SQL statement is translated into native code that introduces new branches and cycles in the controlflow graph.…”

“…This requires time and significantly increases the number of paths to be explored, as each SQL statement is translated into native code that introduces new branches and cycles in the controlflow graph. Secondly, [23] is built upon the Microsoft Pex tool for constraint-based testing, which makes use of a dynamic path exploration, requiring to effectively run the code for each generated test case. Our research prototype tool makes use of a static path exploration and does not need to run the code, which also has an impact on our performance comparison.…”

“…Two research approaches have been developed so far to handle the presence of SQL statements within the classical code to be symbolically executed. The first approach [22,23] introduces new native variables in the classical code to represent the database content and replaces the SQL queries and modification statements by native code acting on these new variables. Classical symbolic execution can then be applied on this normalised version of the tested code.…”