ReAssert: Suggesting Repairs for Broken Unit Tests

Daniel, Brett; Jagannath, Vilas; Dig, Danny; Marinov, Darko

doi:10.1109/ase.2009.17

Cited by 89 publications

(84 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Another observation is that every participant has some failing tests, whose fix requires changing them. Hence, a tool like ReAssert [9] could have benefited all of the participants, potentially helping to fix more than one third of all failing tests. Nevertheless, only a designated study would show how much of the required changes to the failing tests could be automated by ReAssert.…”

Section: How Frequently Do Developers Fix Failing Tests By Changing Tmentioning

confidence: 99%

“…A test that fails due to the new functionality is considered broken since making it a passing test requires fixing the test itself rather than the application under test. Developers either have to fix the broken tests manually or use recent tools that can fix them automatically [8,9,36]. Figure 6 presents a unit test of a parser.…”

Section: How Frequently Do Developers Fix Failing Tests By Changing Tmentioning

confidence: 99%

“…The data collected by CodingTracker enabled us to answer research questions that could not be answered using VCS data alone. The data reveals that 40% of test fixes involve changes to the tests, which motivates the need for automated test fixing tools [8,9,36]. In addition, 24% of changes committed to VCS are untested.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Is It Dangerous to Use Version Control Histories to Study Source Code Evolution?

Negara

Vakilian

Chen

et al. 2012

ECOOP 2012 – Object-Oriented Programming

Self Cite

View full text Add to dashboard Cite

Abstract. Researchers use file-based Version Control System (VCS) as the primary source of code evolution data. VCSs are widely used by developers, thus, researchers get easy access to historical data of many projects. Although it is convenient, research based on VCS data is incomplete and imprecise. Moreover, answering questions that correlate code changes with other activities (e.g., test runs, refactoring) is impossible. Our tool, CodingTracker, non-intrusively records fine-grained and diverse data during code development. CodingTracker collected data from 24 developers: 1,652 hours of development, 23,002 committed files, and 314,085 testcase runs. This allows us to answer: How much code evolution data is not stored in VCS? How much do developers intersperse refactorings and edits in the same commit? How frequently do developers fix failing tests by changing the test itself? How many changes are committed to VCS without being tested? What is the temporal and spacial locality of changes?

show abstract

Section: How Frequently Do Developers Fix Failing Tests By Changing Tmentioning

confidence: 99%

Section: How Frequently Do Developers Fix Failing Tests By Changing Tmentioning

confidence: 99%

See 1 more Smart Citation

Is It Dangerous to Use Version Control Histories to Study Source Code Evolution?

Negara

Vakilian

Chen

et al. 2012

ECOOP 2012 – Object-Oriented Programming

Self Cite

View full text Add to dashboard Cite

show abstract

“…As these existing tests are developed for the old version of the software, when some of them cause failures of the new version in regression testing, the failures may not be due to bugs introduced in the modifications [36,8]. That is to say, some failures are due to bugs in the modified source code under test, but other failures are due to the obsoleteness 3 of some tests.…”

Section: Introductionmentioning

confidence: 99%

“…Furthermore, if developers take obsolete tests as bugs in the software, they may submit some false bug reports and thus incur extra burdens for bug-report processing (e.g., triaging [54] regression testing process but also can help reduce the cost of code modification to fix bugs or repair tests. If the failure is due to a bug, developers can use automated techniques (e.g., [52,56]) to decrease the cost of debugging in the product code; if the failure is due to an obsolete test, developers can also use automated techniques (e.g., [6][7][8]) to decrease the cost of test repair in the test code. It should be noted that techniques for automated debugging typically assume tests to be correct and focus on the product code to fix bugs.…”

Section: Introductionmentioning

confidence: 99%

Is This a Bug or an Obsolete Test?

Hao

Lan

Zhang

et al. 2013

ECOOP 2013 – Object-Oriented Programming

View full text Add to dashboard Cite

Abstract. In software evolution, developers typically need to identify whether the failure of a test is due to a bug in the source code under test or the obsoleteness of the test code when they execute a test suite. Only after finding the cause of a failure can developers determine whether to fix the bug or repair the obsolete test. Researchers have proposed several techniques to automate test repair. However, test-repair techniques typically assume that test failures are always due to obsolete tests. Thus, such techniques may not be applicable in real world software evolution when developers do not know whether the failure is due to a bug or an obsolete test. To know whether the cause of a test failure lies in the source code under test or in the test code, we view this problem as a classification problem and propose an automatic approach based on machine learning. Specifically, we target Java software using the JUnit testing framework and collect a set of features that may be related to failures of tests. Using this set of features, we adopt the Best-first Decision Tree Learning algorithm to train a classifier with some existing regression test failures as training instances. Then, we use the classifier to classify future failed tests. Furthermore, we evaluated our approach using two Java programs in three scenarios (within the same version, within different versions of a program, and between different programs), and found that our approach can effectively classify the causes of failed tests.

show abstract

An Experimental Study on Flakiness and Fragility of Randoop Regression Test Suites

Paydar¹,

Azamnouri²

2019

Fundamentals of Software Engineering

View full text Add to dashboard Cite

Randoop is a well-known tool that proposes a feedbackdirected algorithm for automatic and random generation of unit tests for a given Java class. It automatically generates two test suites for the class under test: 1) an error-revealing test suite, and 2) a regression test suite. Despite successful experiences with applying Randoop on real world projects like Java Development Kit (JDK) which have led to creation of error-revealing tests and identification of real bugs, it has not been investigated in the literature how useful are the regression test suites generated by Randoop. In this paper, we have investigated flakiness and fragility of Randoop's regression tests during evolution of 5 open source Java projects with a total of 78 versions. The results demonstrate that the flakiness of the regression tests is not generally noticeable, since in our dataset, only 5% of the classes have at least one flaky regression tests. In addition, test fragility analysis reveals that in most versions of the projects under study, the regression tests generated by Randoop could be successfully executed on many of later versions. Actually, for 2 out of 5 projects in the experiments, the regression tests generated for each version could be successfully executed on all the later versions of the project.

show abstract

ReAssert: Suggesting Repairs for Broken Unit Tests

Cited by 89 publications

References 47 publications

Is It Dangerous to Use Version Control Histories to Study Source Code Evolution?

Is It Dangerous to Use Version Control Histories to Study Source Code Evolution?

Is This a Bug or an Obsolete Test?

An Experimental Study on Flakiness and Fragility of Randoop Regression Test Suites

Contact Info

Product

Resources

About