Automated testing of refactoring engines

Daniel, Brett; Dig, Danny; Garcia, Kely; Marinov, Darko

doi:10.1145/1287624.1287651

Cited by 175 publications

(139 citation statements)

References 16 publications

Supporting

Mentioning

138

Contrasting

Order By: Relevance

“…Case Study: We implemented our three new techniques in the ASTGen framework for bounded-exhaustive testing of refactoring engines [5]. We chose ASTGen for three reasons: it had enabled finding actual faults in real large software (we had found a few dozens of new faults in the refactoring engines of Eclipse and NetBeans, two popular IDEs for Java [5]); we were familiar with the framework; and we personally experienced the costs of using ASTGen.…”

Section: Oracle-based Test Clustering (Otc)mentioning

confidence: 99%

“…We chose ASTGen for three reasons: it had enabled finding actual faults in real large software (we had found a few dozens of new faults in the refactoring engines of Eclipse and NetBeans, two popular IDEs for Java [5]); we were familiar with the framework; and we personally experienced the costs of using ASTGen. We evaluated the techniques on testing 6 refactorings with 9 generators (explained later in the text).…”

Section: Oracle-based Test Clustering (Otc)mentioning

confidence: 99%

“…One approach to reducing the cost is to automate testing. Bounded-exhaustive testing is an automated approach that checks the code under test for all inputs within given bounds [2,3,4,5,6]. The rationale is that many faults can be revealed within small bounds [7,8], and exhaustively testing within the bounds ensures that no "corner case" is missed.…”

Section: Introductionmentioning

confidence: 99%

“…The rationale is that many faults can be revealed within small bounds [7,8], and exhaustively testing within the bounds ensures that no "corner case" is missed. Bounded-exhaustive testing has been used in both academia and industry to test several real-world applications, with some recent examples including testing of refactoring engines [5] and a web-traversal code [6].…”

Section: Introductionmentioning

confidence: 99%

“…Two key costs in this context are machine time for test generation and execution (which also translates into human time for waiting for these results [9,10]) and human time for inspection of failures. Previous experience shows that bounded-exhaustive testing can discover important faults [3,4,5,11] but also can have high costs. This paper proposes, and evaluates on a case study, three novel techniques that reduce these costs of bounded-exhaustive testing.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Reducing the Costs of Bounded-Exhaustive Testing

Jagannath

Lee

Daniel

et al. 2009

Fundamental Approaches to Software Engineering

Self Cite

View full text Add to dashboard Cite

Abstract. Bounded-exhaustive testing is an automated testing methodology that checks the code under test for all inputs within given bounds: first the user describes a set of test inputs and provides test oracles that check test outputs; then the tool generates all the inputs, executes them on the code under test, and checks the outputs; and finally the user inspects failing tests to submit bug reports. The costs of boundedexhaustive testing include machine time for test generation and execution (which translates into human time waiting for these results) and human time for inspection of results. This paper proposes three techniques that reduce these costs. Sparse Test Generation skips some tests to reduce the time to the first failing test. Structural Test Merging generates a smaller number of larger test inputs (rather than a larger number of smaller test inputs) to reduce test generation and execution time. Oracle-based Test Clustering groups failing tests to reduce the inspection time. Results obtained from the bounded-exhaustive testing of the Eclipse refactoring engine show that these three techniques can substantially reduce the costs while mostly preserving fault-detection capability.

show abstract

Section: Oracle-based Test Clustering (Otc)mentioning

confidence: 99%

Section: Oracle-based Test Clustering (Otc)mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Reducing the Costs of Bounded-Exhaustive Testing

Jagannath

Lee

Daniel

et al. 2009

Fundamental Approaches to Software Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

Clone refactoring inspection by summarizing clone refactorings and detecting inconsistent changes during software evolution

Chen

Kwon

Song

2018

J Software Evolu Process

View full text Add to dashboard Cite

It has been broadly assumed that removing code clones by refactorings would solve the problems of code duplication. Despite recent empirical studies on the benefit of refactorings, contradicting evidence shows that it is often difficult or impossible to remove clones by using standard refactoring techniques. Developers cannot easily determine which clones can be refactored or how they should be maintained scattered throughout a large code base in evolving systems. We propose pattern‐based clone refactoring inspection (PRI), a technique for managing clone refactorings. PRI summarizes refactorings of clones and detects clones that are not consistently refactored. To help developers refactor these anomalies, PRI also visualizes clone evolution and refactorings and fixes refactoring anomalies to prevent the clone group from being left in an inconsistent state. We evaluated PRI on 6 open‐source projects and showed that it identifies clone refactorings with 94.1% accuracy and detects inconsistent refactorings with 98.4% accuracy, tracking clone change histories. In a study with 10 student developers, the participants reported that flexible PRI's summarization and detection features can be valuable for novice developers to learn about refactorings to clones. These results show that PRI should improve developer productivity in inspecting clone refactorings distributed across multiple files in evolving systems.

show abstract

RUGRAT: Evaluating program analysis and testing tools and compilers with large generated random benchmark applications

Hussain

Csallner

Grechanik

et al. 2014

Softw. Pract. Exper.

View full text Add to dashboard Cite

Summary Benchmarks are heavily used in different areas of computer science to evaluate algorithms and tools. In program analysis and testing, open‐source and commercial programs are routinely used as benchmarks to evaluate different aspects of algorithms and tools. Unfortunately, many of these programs are written by programmers who introduce different biases, not to mention that it is very difficult to find programs that can serve as benchmarks with high reproducibility of results. We propose a novel approach for generating random benchmarks for evaluating program analysis and testing tools and compilers. Our approach uses stochastic parse trees, where language grammar production rules are assigned probabilities that specify the frequencies with which instantiations of these rules will appear in the generated programs. We implemented our tool for Java and applied it to generate a set of large benchmark programs of up to 5M lines of code each with which we evaluated different program analysis and testing tools and compilers. The generated benchmarks let us independently rediscover several issues in the evaluated tools. Copyright © 2014 John Wiley & Sons, Ltd.

show abstract

Automated testing of refactoring engines

Cited by 175 publications

References 16 publications

Reducing the Costs of Bounded-Exhaustive Testing

Reducing the Costs of Bounded-Exhaustive Testing

Clone refactoring inspection by summarizing clone refactorings and detecting inconsistent changes during software evolution

RUGRAT: Evaluating program analysis and testing tools and compilers with large generated random benchmark applications

Contact Info

Product

Resources

About