Impacts of Data Uniformity in the Reuse of Acceptance Test Glue Code

Abstract

This paper presents the design and results of an experiment to evaluate the impact/effect of data uniformity in automation of acceptance tests. An experiment to specify acceptance tests, represented by the User Scenarios through User Interaction Diagrams (US-UIDs) format, with non-technical user has been set up involving two projects. In the first project, called P1, the treatment of data uniformity is held by an expert, while in the second project, called P2, no treatment of data uniformity is done. In both projects, automation of acceptance tests was developed for evaluation and comparison of the following artifacts: data uniformity, fixture name sharing, automation time, and glue code volume. The results show that there is a meaningful statistics difference of uniformity between projects P1 and P2, where P1 resulted in a better uniformity. However, although the treatment of data uniformity does not show meaningful statistics difference according to the strategy of fixture names sharing used, the time spent in fixture naming was more than two times higher in P2. In addition, the glue code volume was less than half in P1 comparing to P2.

Measuring Test Data Uniformity in Acceptance Tests for the FitNesse and Gherkin Notations

“…Yet, none of them have proposed metrics for data uniformity that are specific to FitNesse and Gherkin notations. In other words, to the best of our knowledge, no other objective metrics to assess the uniformity of acceptance test data has been found, other than (Longo and Vilain, 2018;Longo et al, 2019), especially for FitNesse and Gherkin techniques. In addition, there is lack of basic studies comparing the different notations of automated acceptance tests that consider a large volume of projects.…”

“…The two main related works are (Longo and Vilain, 2018;Longo et al, 2019). Longo and Vilain (2018) propose a kind of metric for measuring data uniformity in automated acceptance tests in the notation of User Scenarios through User Interaction Diagrams (US-UIDs).…”

“…Longo and Vilain (2018) propose a kind of metric for measuring data uniformity in automated acceptance tests in the notation of User Scenarios through User Interaction Diagrams (US-UIDs). Longo et al (2019), the authors elaborate an experiment with the treatment of data uniformity as the control factor. The conclusions were that with the treatment of data uniformity, both the required volume of glue code and the time spent to automate the tests were reduced.…”

“…Test data are represented by words or expressions, usually with a flag to differentiate it from functional data. Longo and Vilain (2018;Longo et al, 2019) define that the test data can be either uniform or irregular. Uniform test data are expressions that are common to various test documents and irregular test data are composed by single expressions that are not repeated through test documents.…”

“…For those involved in specifying a test, communication between tests using irregular data may be feasible, with little or no information loss, since humans are able to interpret the irregularities of test data and understand the meaning of the test. However, there may be problems associated with unintentionally irregular data and glue code reuse for test automation (Longo et al, 2019). For example, in order to automate the scenarios of the login functionality feature (Fig.…”