A systematic literature review of test breakage prevention and repair techniques

Abstract

Context: When an application evolves, some of the developed test cases break. Discarding broken test cases causes a significant waste of effort and leads to test suites that are less effective and have lower coverage. Test repair approaches evolve test suites along with applications by repairing the broken test cases.Objective: Numerous studies are published on test repair approaches every year. It is important to summarise and consolidate the existing knowledge in the area to provide directions to researchers and practitioners. This research work provides a systematic literature review in the area of test case repair and breakage prevention, aiming to guide researchers and practitioners in the field of software testing. Method:We followed the standard protocol for conducting a systematic literature review. First, research goals were defined using the Goal Question Metric (GQM). Then we formulate research questions corresponding to each goal. Finally, metrics are extracted from the included papers. Based on the defined selection criteria a final set of 41 primary studies are included for analysis. Results:The selection process resulted in 5 journal papers, and 36 conference papers. We present a taxonomy that lists the causes of test case breakages extracted from the literature. We found that only four proposed test repair tools are publicly available. Most studies evaluated their approaches on open-source case studies. Conclusion:There is significant room for future research on test repair techniques. Despite the positive trend of evaluating approaches on large scale open source studies, there is a clear lack of results from studies done in a real industrial context. Few tools are publicly available which lowers the potential of adaption by industry practitioners.

A Systematic Literature Review of Deep Learning-Based Detection and Classification Methods for Bacterial Colonies

“…This study presents an SLR by following the guidelines presented by [16] [17]. These guidelines are well accepted in the software engineering community and have been followed by a number of studies [18] [19]. Fig.…”

Approach of Agile Methodologies in the Development of Web-Based Software

“…Two research questions have been defined to guide the search of the literature in databases and to allow us to address the main scope of the document. The methodological process of searching within this research was based on the search models applied in the investigation of Javaria et al [17] and Anureet & Kulwant [18] to encompass the purpose of the study.…”

AdapTV: A Model-Based Test Adaptation Approach for End-to-End User Interface Testing of Smart TVs

“…Needless to say, manually adapting the broken test cases is tedious, error-prone, and costly [2]- [4], thus not sustainable at all. It, however, turns out that automating the test adaptation process in the TV domain (and in the consumer electronics domain in general) has its own specific challenges on top of all the other challenges of test adaptation in the related domains, including the mobile and web application domains [5]- [7]. One challenge is that, although an integral part of test adaptation is to figure out what is actually on the screen, smart TVs, unlike the web and mobile platforms, do not necessarily provide object models for their screens, such as Page Object Models (POMs) or Document Object Models (DOMs), which report many useful attributes for the UI elements present on the screens, including their types, locations, and labels [8]- [11].…”