DOMINO: Fast and Effective Test Data Generation for Relational Database Schemas

Abstract

Abstract-An organization's databases are often one of its most valuable assets. Data engineers commonly use a relational database because its schema ensures the validity and consistency of the stored data through the specification and enforcement of integrity constraints. To ensure their correct specification, industry advice recommends the testing of the integrity constraints in a relational schema. Since manual schema testing is laborintensive and error-prone, this paper presents DOMINO, a new automated technique that generates test data according to a coverage criterion for integrity constraint testing. In contrast to more generalized search-based approaches, which represent the current state of the art for this task, DOMINO uses tailored, domain-specific operators to efficiently generate test data for relational database schemas. In an empirical study incorporating 34 relational database schemas hosted by three different database management systems, the results show that DOMINO can not only generate test suites faster than the state-of-the-art search-based method but that its test suites can also detect more schema faults.

What Factors Make SQL Test Cases Understandable for Testers? A Human Study of Automated Test Data Generation Techniques

“…Since it may be challenging for testers to identify attribute values that are tailored to the constraints in a schema, test data generation techniques support this process by automatically creating tests that aim to cover all the integrity constraints. For schemas, there are many techniques for test data generation, such as Random + [10], the Alternating Variable Method (AVM) [4], [10], and DOMINO [6], [17], with prior work showing that AVM and DOMINO are often the most effective [6].…”

“…Operating with the same goal as the AVM, DOMINO is a random technique that uses domain-specific operators to generate test data for schema testing. After creating random values, it uses operators that perform value copying and randomization, the setting or removal of NULLs, and the solving of CHECK constraints [6]. The copying operator helps to match values depending on the coverage criteria (e.g., it copies values to violate a UNIQUE constraint or to link a FOREIGN KEY).…”

“…The randomizing operator aids when DOMINO must generate distinct values to satisfy the coverage criteria. When solving CHECKs, DOMINO uses randomization to select values from a constant pool containing values mined from the schema [6]. Figure 1(b) gives examples of tests, produced by both aforementioned techniques, that violate the UNIQUE constraint of the cookies table.…”

“…DOM-RND is the basic form of DOMINO, which a prior study found to obtain the highest mutation scores out of all studied testing methods [6]. Figure 2(b) gives an example of a test in which this method generated all values randomly or randomly selected from constants mined from the schema.…”

“…Since schemas often contain many tables and integrity constraints, manually writing most of these tests is tedious and error-prone. As such, prior work presented automated test data generators that output a test suite of INSERTs that effectively covers the test requirements [5], [6].…”