More Reliable Test Suites for Dynamic APR by using Counterexamples

Nilizadeh, Amirfarhad; Calvo, Marlon; Leavens, Gary T.; Le, Xuan-Bach D.

doi:10.1109/issre52982.2021.00032

Cited by 12 publications

(3 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, by running the APR, they can generate more reliable patches (A. Nilizadeh, Calvo, et al 2021;Huang & Meyer 2022;A. Nilizadeh 2021b).…”

Section: Patch Correctness Assessment For Aprmentioning

confidence: 99%

Toward Using Fuzzers and Lightweight Specifications to Reveal Semantic Bugs.

Nilizadeh,

Leavens,

Cok

2024

JOT

View full text Add to dashboard Cite

Although fuzzers have been successful in revealing semantic bugs that lead to crashes, they do not reveal semantic bugs that do not lead to crashes. Furthermore, the inputs that lead to crashes may be invalid and invalid inputs do not reveal semantic bugs at all, since they are outside the program's intended input domain. On the other hand, runtime assertion checking (RAC) may be used for revealing semantic bugs, although it needs input test data that can trigger these bugs.In this idea paper, we propose the idea of combining different kinds of fuzzing tools and RAC in a complementary manner to leverage their benefits and overcome these problems, along with a preliminary study. That is, a fuzzing tool will generate an input test, and a RAC tool will make sure that the generated inputs are valid and check the results for semantic bugs.

show abstract

“…Then, by running the APR, they can generate more reliable patches (A. Nilizadeh, Calvo, et al 2021;Huang & Meyer 2022;A. Nilizadeh 2021b).…”

Section: Patch Correctness Assessment For Aprmentioning

confidence: 99%

Toward Using Fuzzers and Lightweight Specifications to Reveal Semantic Bugs.

Nilizadeh,

Leavens,

Cok

2024

JOT

View full text Add to dashboard Cite

show abstract

“…The goal is to express (NVC) on Line 42. The previous lines define the environment in the form of declarations of types or 'sorts' (Lines 1 and 2), functions and constants 3 (3-19), assertions expressing typing properties (20)(21)(22)(23) and assertions expressing verification conditions (24)(25)(26)(27)(28)(29)(30)(31)(32)(33)(34)(35)(36)(37)(38)(39)(40)(41)(42). The final line, check-sat, directs the solver to check satisfaction of the conditions.…”

Section: Boogie and Z3mentioning

confidence: 99%

“…The idea of using counterexamples to generate test cases is not new, but it has been mostly applied to verification approaches using model checking of temporal-logic specifications [17][18][19][20]. We are only aware of one existing attempt [21] (building on work on using counterexamples for automatic program repair [22]) to apply the idea in the context of Hoare-style verification; it exploits counterexamples produced by OpenJML [23] (a verification tool for Java programs) to generate unit tests in JUnit [24] format. The tool described in that article needs both to generate counterexamples from an SMT server and to generate, from the JML source, Java code instrumented to monitor some of the assertions (expressed as comments in the original code) at run-time.…”

Section: Related Workmentioning

confidence: 99%

A failed proof can yield a useful test

Huang,

Meyer

2023

Software Testing Verif & Rel

View full text Add to dashboard Cite

A successful automated program proof is, in software verification, the ultimate triumph. In practice, however, the road to such success is paved with many failed proof attempts. Unlike a failed test, which provides concrete evidence of an actual bug in the program, a failed proof leaves the programmer in the dark. Can we instead learn something useful from it? The work reported here takes advantage of the rich information that some automatic provers internally collect about the program when attempting a proof. If the proof fails, the Proof2Test tool presented in this article uses the counterexample generated by the prover (specifically, the SMT solver underlying the Boogie tool used in the AutoProof system to perform correctness proofs of contract‐equipped Eiffel programs) to produce a failed test, which provides the programmer with immediately exploitable information to correct the program. The discussion presents Proof2Test and the application of the ideas and tool to a collection of representative examples.

show abstract

Better Counterexamples for Dafny

Chakarov

Fedchin

Rakamarić

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Dafny is a verification-aware programming language used at Amazon Web Services to develop critical components of their access management, storage, and cryptography infrastructures. The Dafny toolchain provides a verifier that can prove an implementation of a method satisfies its specification. When the underlying SMT solver cannot establish a proof, it generates a counterexample. These counterexamples are hard to understand and their interpretation is often a bottleneck in the proof debugging process. In this paper, we introduce an open-source tool that transforms counterexamples generated by the SMT solver to a more user-friendly format that maps to the Dafny syntax and is suitable for further processing. This new tool allows the Dafny developers to quickly identify the root cause of a problem with their proof, thereby speeding up the development of Dafny projects.

show abstract

More Reliable Test Suites for Dynamic APR by using Counterexamples

Cited by 12 publications

References 71 publications

Toward Using Fuzzers and Lightweight Specifications to Reveal Semantic Bugs.

Toward Using Fuzzers and Lightweight Specifications to Reveal Semantic Bugs.

A failed proof can yield a useful test

Better Counterexamples for Dafny

Contact Info

Product

Resources

About