Testing database transactions with AGENDA

Deng, Yuetang; Frankl, Phyllis G.; Chays, David

doi:10.1145/1062455.1062486

Cited by 59 publications

(46 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The bug-catching task becomes even harder when the existing testing methodologies for transaction processing only validate functional requirements [6,7]. The validation of non-functional requirements is still an open issue and strongly rely on the combination of unit testing tools 4 (e.g., Jepsen, JUnit, Jmeter, PeerUnit [8]) and benchmarks to reproduce specific workloads (e.g., TPC-like, YCSB).…”

Section: Motivationmentioning

confidence: 99%

“…Jepsen 6 , Hammerora 7 , AppPerfect 8 and Oracle Application Testing Suite 9 provide a test driver to build up test cases on top of TPC-like benchmarks. Agenda [7] provides its own methodology and test driver, but this tool can only generate functional test cases. JMeter is also a well known and widely applied load testing tool for different applications, including DBMS.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

“Overloaded!” — A Model-Based Approach to Database Stress Testing

Meira

Almeida

Kim

et al. 2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. As a new era of "Big Data" comes, contemporary database management systems (DBMS) introduced new functions to satisfy new requirements for big volume and velocity applications. Although the development agenda goes at full pace, the current testing agenda does not keep up, especially to validate non-functional requirements, such as: performance and scalability. The testing approaches strongly rely on the combination of unit testing tools and benchmarks. There is still a testing methodology missing, in which testers can model the runtime environment of the DBMS under test, defining the testing goals and the harness support for executing test cases. The major contribution of this paper is the MoDaST (Model-based Database Stress Testing) approach that leverages a state transition model to reproduce a runtime DBMS with dynamically shifting workload volumes and velocity. Each state in the model represents the possible running states of the DBMS. Therefore, testers can define state goals or specific state transitions that revealed bugs. Testers can also use MoDaST to pinpoint the conditions of performance loss and thrashing states. We put MoDaST to practical application testing two popular DBMS: PostgreSQL and VoltDB. The results show that MoDaST can reach portions of source code that are only possible with non-functional testing. Among the defects revealed by MoDaST, when increasing the code coverage, we highlight a defect confirmed by the developers of VoltDB as a major bug and promptly fixed.

show abstract

Section: Motivationmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

“Overloaded!” — A Model-Based Approach to Database Stress Testing

Meira

Almeida

Kim

et al. 2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…On a related but complementary level, a substantial amount of work (e.g. [44,45,46,47,33,48,49,50,51,52,20,53,54,55,56]) has been done on how to generate test database content exhibiting some desirable properties, given only the database schema and possibly some queries to be executed over the database. The main difference between our work and these approaches is that they essentially work without considering the control flow of the programs manipulating the database.…”

Section: Other Related Workmentioning

confidence: 99%

Relational symbolic execution of SQL code for unit testing of database programs

Marcozzi

Vanhoof

Hainaut

2015

Science of Computer Programming

View full text Add to dashboard Cite

Symbolic execution is a technique enabling the automatic generation of test inputs that exercise a set of execution paths within a code unit to be tested. If the paths cover a sufficient part of the code under test, the test data offer a representative view of the actual behaviour of this code. This notably enables detecting errors and correcting faults. Relational databases are ubiquitous in software, but symbolic execution of code units that manipulate them remains a non-trivial problem, particularly because of the complex structure of such databases and the complex behaviour of SQL statements. Finding errors in such code units is yet critical, as it can avoid corrupting important data. In this work, we define a symbolic execution translating database manipulation code directly into constraints and integrate it with a more traditional symbolic execution of normal program code. The database tables are represented by relational symbols and the SQL statements by relational constraints over these symbols. An algorithm based on these principles is presented for the symbolic execution of simple Java methods that implement transactional use cases by reading and writing in a relational database, the latter subject to data integrity constraints. The algorithm is integrated in a test generation tool and experimented over sample code. The target language for the constraints produced by the tool is the SMT-Lib standard and the used solver is Microsoft Z3. The results show that the proposed approach enables generating meaningful test data, including valid database content, in reasonable time. In particular, the Z3 solver is shown to be more scalable than the Alloy solver, used in our previous work, for solving relational constraints.

show abstract

“…Relational algebra is used in [30,50] and general purpose constraint solvers for test data generation in [58]. The information on the database schema and the SQL queries is completed with heuristics supplied by the tester to automatically generate test cases and checking the test results in the AGENDA tool [11], which has been extended to deal with transactions [15]. The test cases are specified in [54] using pre and post-conditions in the form of intensional rules (using an SQL-like language) and then the database populated for fulfilling the rules.…”

Section: Related Work On Testing Database Applicationsmentioning

confidence: 99%

“…A mutation approach was used in [50] to perform a partial evaluation of the fault detection capability of database test cases and in [15,17] for seeding manual faults in queries in order to assess the effectiveness of test generation techniques.…”

Section: Mutation Testing Overviewmentioning

confidence: 99%

Mutating database queries

Tuya

Suárez-Cabal

Riva

2007

Information and Software Technology

102

View full text Add to dashboard Cite

Abstract.A set of mutation operators for SQL queries that retrieve information from a database is developed and tested against a set of queries drawn from the NIST SQL Conformance Test Suite. The mutation operators cover a wide spectrum of SQL features, including the handling of null values. Additional experiments are performed to explore whether the cost of executing mutants can be reduced using selective mutation or the test suite size can be reduced by using an appropriate ordering of the mutants. The SQL mutation approach can be helpful in assessing the adequacy of database test cases and their development, and as a tool for systematically injecting faults in order to compare different database testing techniques.

show abstract

Testing database transactions with AGENDA

Cited by 59 publications

References 21 publications

“Overloaded!” — A Model-Based Approach to Database Stress Testing

“Overloaded!” — A Model-Based Approach to Database Stress Testing

Relational symbolic execution of SQL code for unit testing of database programs

Mutating database queries

Contact Info

Product

Resources

About