Scented since the beginning: On the diffuseness of test smells in automatically generated test code

Abstract

Software testing represents a key software engineering practice to ensure source code quality and reliability. To support developers in this activity and reduce testing effort, several automated unit test generation tools have been proposed. Most of these approaches have the main goal of covering as more branches as possible. While these approaches have good performance, little is still known on the maintainability of the test code they produce, i.e., whether the generated tests have a good code quality and if they do not possibly introduce issues threatening their effectiveness. To bridge this gap, in this paper we study to what extent existing automated test case generation tools produce potentially problematic test code. We consider seven test smells, i.e., suboptimal design choices applied by programmers during the development of test cases, as measure of code quality of the generated tests, and evaluate their diffuseness in the unit test classes automatically generated by three state-of-the-art tools such as Randoop, JTExpert, and Evosuite. Moreover, we investigate whether there are characteristics of test and production code influencing the generation of smelly tests. Our study shows that all the considered tools tend to generate a high quantity of two specific test smell types, i.e., Assertion Roulette and Eager Test, which are those that previous studies showed to negatively impact the reliability of production code. We also discover that test size is correlated with the generation of smelly tests. Based on our findings, we argue that more effective automated generation algorithms that explicitly take into account test code quality should be further investigated and devised.

Pizza versus Pinsa: On the Perception and Measurability of Unit Test Code Quality

“…This represents a crucial challenge for the research community. While our findings motivate the growing research area around test code design (e.g., how to best generate tests automatically [72][73][74]), we believe that further efforts should put in place for the definition of best and bad practices that can help developers writing high quality unit tests, e.g., novel test code design patterns.…”

Implementation of Randoop Automated Test Generation Tool for Novel Perspective on Software Development

“…The set of sequences which are valid and break no exception are then randomly selected and added to sequence S. The machine flow should add all of the specific s to the sequence S. Multiple calls to m1 are required in some specific cases to attain the required object state. Randoop induces mechanism to iterate for this particular purpose and for this reason it adds M calls with a default highest value equal to 100 instead of adding a single call m1 to create a new sequence [4], [3]. When the search get over Randoop generates the regression test from S, this happens when the stated or default time period is over.…”

“…The functional characteristics (example accuracy) and extra-functional characteristics (example efficiency) of the methods are subject to industrial requirements and the quality of code must be guaranteed in today's data driven environment, and the impact of method input parameters must be considered in the analysis of those properties [1]. The primary intention is to enhance code coverage and trim 1.the time of execution 2.cost 3.dynamic memory usage 4.Number of mistakenly missed goals [4]. Unit testing has been one of the leading software testing methods with the main goal of evaluating the software's individual programming element for example, several method in the source code, a list of certain of methods of the system, or classes along with suitable control over data and operating steps.…”

“…Randoop implies that random feedback generation retains the advantages of random testing (scalability, ease of implementation), eliminates the drawbacks of random testing (generation of inefficient or inconsequential input data), and is able to compete with systematic techniques [6]. There is still a shortage of factual analysis for the quality of the test code generated by such instruments [4]. The aim of such tests is to address complicated faults, however there are no autonomous oracles and which have to be inspected manually after the creation process.…”

“…Moreover, recent studies have already shown that badly written test scripts (i) causes quite-called flakiness, i.e. the non -trivial actions of a test (ii) restricts the ability of test cases to detect faults [4].…”