Generating unit tests with descriptive names or: would you name your children thing1 and thing2?

Abstract

The name of a unit test helps developers to understand the purpose and scenario of the test, and test names support developers when navigating amongst sets of unit tests. When unit tests are generated automatically, however, they tend to be given non-descriptive names such as "test0", which provide none of the benets a descriptive name can give a test. The underlying challenge is that automatically generated tests typically do not represent real scenarios and have no clear purpose other than covering code, which makes naming them dicult. In this paper, we present an automated approach which generates descriptive names for automatically generated unit tests by summarizing API-level coverage goals. The tests are optimized to be short, descriptive of the test, have a clear relation to the covered code under test, and allow developers to uniquely distinguish tests in a test suite. An empirical evaluation with 47 participants shows that developers agree with the synthesized names, and the synthesized names are equally descriptive as manually written names. Study participants were even more accurate and faster at matching code and tests with synthesized names compared to manually derived names. CCS CONCEPTS •Software and its engineering ! Software maintenance tools; Software testing and debugging;

Revisiting Test Smells in Automatically Generated Tests: Limitations, Pitfalls, and Opportunities

“…To further improve maintainability, these assertions can be minimized for their fault-finding potential [21]. Further optimizations for maintainability include applying specific heuristics from industrial application [22], semantic simplification [23], prioritizing using literals taken from the source code [24], generation of meaningful names for tests [25], measuring readability using prediction models [26], and adding textual summaries [27].…”

Test Case Understandability Model

“…The understandability criterion is important because test cases that are generated automatically have a very long format and are difficult to read [6]. Several studies have proposed improving the understanding of test cases by generating documentation in the form of natural language that contains summaries of test cases [41], simplifying test cases from the size of the LOC [40], and giving the name of the identifier that describes contents of test cases [42]. Understandability measurement is often associated with readability criteria because it is assumed that code is easy to read will be easy to understand.…”

“…To improve automated test case presentation, Daka optimized the search-based generation test case by using readability evaluation as a secondary fitness function []. Daka has also evaluated whether the readability optimized test case can be better understood by the developer or not [6], [42]. Evaluation of the model was conducted by giving some questions related to the understanding of test cases to 30 students.…”

What Factors Make SQL Test Cases Understandable for Testers? A Human Study of Automated Test Data Generation Techniques

“…Test Understandability Factors. Although human oracle costs can be ameliorated by creating automated test data generation methods that consider readability (e.g., [13], [22]), to the best of our knowledge there is no prior work aiming to characterize and limit the qualitative human oracle costs associated with the automated testing of a database schema. As a first step, we must determine how generated test data affects a human's understanding of a test's behavior.…”

“…Grano et al reported that manually written tests are more readable than automated tests [40]. Different approaches have been created to address this issue: test visualization [41], [42], automatic test documentation [38], [43], [44], and test readability improvement [14], [15], [22].…”