The impact of test case summaries on bug fixing performance

Panichella, Sebastiano; Panichella, Annibale; Beller, Moritz; Zaidman, Andy; Gall, Harald C.

doi:10.1145/2884781.2884847

Cited by 77 publications

(13 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Search-based unit test generation has been extensively investigated by prior studies [13,15,29]. These studies have confirmed that it achieves a high level of coverage [15,29], detects faults in real-world applications [2,16], and reduces the debugging costs [30]. Most search-based unit test generation approaches abstract the source code of a method to a control flow graph: Definition 1 (Control Flow Graph (CFG) [1]).…”

Section: Search-based Unit Test Generationmentioning

confidence: 99%

Commonality-Driven Unit Test Generation

Evers

Derakhshanfar

Devroey

et al. 2020

Search-Based Software Engineering

Self Cite

View full text Add to dashboard Cite

Various search-based test generation techniques have been proposed to automate the generation of unit tests fulfilling different criteria (e.g., line coverage, branch coverage, mutation score, etc.). Despite several advances made over the years, search-based unit test generation still suffers from a lack of guidance due to the limited amount of information available in the source code that, for instance, hampers the generation of complex objects. Previous studies introduced many strategies to address this issue, e.g., dynamic symbolic execution or seeding, but do not take the internal execution of the methods into account. In this paper, we introduce a novel secondary objective called commonality score, measuring how close the execution path of a test case is from reproducing a common or uncommon execution pattern observed during the operation of the software. To assess the commonality score, we implemented it in EvoSuite and evaluated its application on 150 classes from JabRef, an open-source software for managing bibliography references. Our results are mixed. Our approach leads to test cases that indeed follow common or uncommon execution patterns. However, if the commonality score can have a positive impact on the structural coverage and mutation score of the generated test suites, it can also be detrimental in some cases.

show abstract

Section: Search-based Unit Test Generationmentioning

confidence: 99%

Commonality-Driven Unit Test Generation

Evers

Derakhshanfar

Devroey

et al. 2020

Search-Based Software Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…Moreno et al have developed Java class-level summarization approaches [8] while others focus on the method level such as Sridhara et al [11], McBurney et al [7], Rastkar et al [10] and Haiduc et al [3]. There is less work translating Java test code all approaches (Zhang et al [12], Kamimura and Murphy [4], Li et al [5] and Panichella et al [9] adapt the template-based source code approaches, and also use similar approaches for distinguishing parameters. However, in no case is such fine-grained attention paid to the conversion of the assert statements, especially the special cases for assertThat.…”

Section: Related Workmentioning

confidence: 99%

“…Many existing works have shown the feasibility and benefits of this approach by summarizing Java methods and classes [3,7,8,10,11]. Recently, these works have been adapted for the summarization of unit test code, and summaries of JUnit test cases have been shown to improve the understandability of test cases [4,5,9,12]. While these test summarization approaches result in readable summaries, they provide heuristics for converting only a limited subset of the complex-but-critical assert statement API.…”

Section: Introductionmentioning

confidence: 99%

A fine-grained approach for automated conversion of JUnit assertions to English

Gonzalez

Prentice

Mirakhorli

2018

Proceedings of the 4th ACM SIGSOFT International Workshop on NLP for Software Engineering

View full text Add to dashboard Cite

Converting source or unit test code to English has been shown to improve the maintainability, understandability, and analysis of software and tests. Code summarizers identify 'important' statements in the source/tests and convert them to easily understood English sentences using static analysis and NLP techniques. However, current test summarization approaches handle only a subset of the variation and customization allowed in the JUnit assert API (a critical component of test cases) which may affect the accuracy of conversions. In this paper, we present our work towards improving JUnit test summarization with a detailed process for converting a total of 45 unique JUnit assertions to English, including 37 previously-unhandled variations of the assertThat method. This process has also been implemented and released as the AssertConvert tool. Initial evaluations have shown that this tool generates English conversions that accurately represent a wide variety of assertion statements which could be used for code summarization or other NLP analyses.

show abstract

“…But their paper expose the main ones. Panichella et al [2016] explained it is difficult to understand the result of generated test. In fact, generated tests are hard to maintain and provide unusable data for "simple" developers.…”

Section: Related Workmentioning

confidence: 99%

Usage of Tests in an Open-Source Community

Verhaeghe

Anquetil

Ducasse

et al. 2017

Proceedings of the 12th Edition of the International Workshop on Smalltalk Technologies

View full text Add to dashboard Cite

During the development, it is known that tests ensure the good behavior of applications and improve their quality. We studied developers testing behavior inside the Pharo community in the purpose to improve it. In this paper, we take inspiration from a paper of the literature to enhance our comprehension of test habits in our open source community. We report results of a field study on how often the developers use tests in their daily practice, whether they make use of tests selection and why they do. Results are strengthened by interviews with developers involved in the study. The main findings are that developers run tests every modifications of their code they did; most of the time they practice test selection (instead of launching an entire test suite); however they are not accurate in their selection; they change their selection depending on the duration of the tests and; contrary to expectation, test selection is not influenced by the size of the test suite.

show abstract

The impact of test case summaries on bug fixing performance

Cited by 77 publications

References 58 publications

Commonality-Driven Unit Test Generation

Commonality-Driven Unit Test Generation

A fine-grained approach for automated conversion of JUnit assertions to English

Usage of Tests in an Open-Source Community

Contact Info

Product

Resources

About