Differentially Testing Database Transactions for Fun and Profit

Cui, Ziyu; Dou, Wensheng; Dai, Qianwang; Song, Jiansen; Wang, Wei; Wei, Jun; Ye, Dan

doi:10.1145/3551349.3556924

Cited by 9 publications

(2 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because consistency-checking algorithms additionally require the overhead for analysis, researchers explored a technique to apply verifiable computation to check the correctness of serializability [66]. Instead of developing a consistency-checking algorithm as an arbitrary invariant checker, differential testing has been employed [67]. Viper has been developed to check for snapshot isolation by checking for a polygraph-based transaction dependency [68].…”

Section: Consistency Checkingmentioning

confidence: 99%

VConMC: Enabling Consistency Verification for Distributed Systems Using Implementation-Level Model Checkers and Consistency Oracles

Kim

2024

Electronics

View full text Add to dashboard Cite

Many cloud services are relying on distributed key-value stores such as ZooKeeper, Cassandra, HBase, etc. However, distributed key-value stores are notoriously difficult to design and implement without any mistakes. Because data consistency is the contract for clients that defines what the correct values to read are for a given history of operations under a specific consistency model, consistency violations can confuse client applications by showing invalid values. As a result, serious consequences such as data loss, data corruption, and unexpected behavior of client applications can occur. Software bugs are one of main reasons why consistency violations may occur. Formal verification techniques may be used to make designs correct and minimize the risks of having bugs in the implementation. However, formal verification is not a panacea due to limitations such as the cost of verification, inability to verify existing implementations, and human errors involved. Implementation-level model checking has been heavily explored by researchers for the past decades to formally verify whether the underlying implementation of distributed systems have bugs or not. Nevertheless, previous proposals are limited because their invariant checking is not versatile enough to check for the wide spectrum of consistency models, from eventual consistency to strong consistency. In this work, consistency oracles are employed for consistency invariant checking that can be used by implementation-level model checkers to formally verify data consistency model implementations of distributed key-value stores. To integrate consistency oracles with implementation-level distributed system model checkers, the partial-order information obtained via API is leveraged to avoid the exhaustive search during consistency invariant checking. Our evaluation results show that, by using the proposed method for consistency invariant checking, our prototype model checker, VConMC, can detect consistency violations caused by several real-world software bugs in a well-known distributed key-value store, ZooKeeper.

show abstract

Section: Consistency Checkingmentioning

confidence: 99%

VConMC: Enabling Consistency Verification for Distributed Systems Using Implementation-Level Model Checkers and Consistency Oracles

Kim

2024

Electronics

View full text Add to dashboard Cite

show abstract

“…RAGS [8] executes the same SELECT query on different DBMSs and observes discrepancies in their query results. DT2 [46] feeds a group of transactions into multiple DBMSs to detect transaction bugs. Grand [45] and RD2 [49] apply differential testing on graph database systems.…”

Section: Related Workmentioning

confidence: 99%

Testing Database Systems via Differential Query Execution

Song

Dou

Cui

et al. 2023

2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)

Self Cite

View full text Add to dashboard Cite

Database Management Systems (DBMSs) provide efficient data retrieval and manipulation for many applications through Structured Query Language (SQL). Incorrect implementations of DBMSs can result in logic bugs, which cause SELECT queries to fetch incorrect results, or UPDATE and DELETE queries to generate incorrect database states. Existing approaches mainly focus on detecting logic bugs in SELECT queries. However, logic bugs in UPDATE and DELETE queries have not been tackled.In this paper, we propose a novel and general approach, which we have termed Differential Query Execution (DQE), to detect logic bugs in SELECT, UPDATE and DELETE queries of DBMSs. The core idea of DQE is that different SQL queries with the same predicate usually access the same rows in a database. For example, a row updated by an UPDATE query with a predicate φ should also be fetched by a SELECT query with the same predicate φ. If not, a logic bug is revealed in the target DBMS. To evaluate the effectiveness and generality of DQE, we apply DQE on five production-level DBMSs, i.e., MySQL, MariaDB, TiDB, CockroachDB and SQLite. In total, we have detected 50 unique bugs in these DBMSs, 41 of which have been confirmed, and 11 have been fixed. We expect that the simplicity and generality of DQE can greatly improve the reliability of DBMSs.

show abstract