A Search-Based Approach for Cost-Effective Software Test Automation Decision Support and an Industrial Case Study

Amannejad, Yasaman; Garousi, Vahid; Irving, Rob; Sahaf, Zahra

doi:10.1109/icstw.2014.34

Cited by 23 publications

(12 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the adoption of automated testing in the Turkish industry is very low, it seems that most Turkish SE practitioners are unaware of the advantages of automated testing. Thus, we recommend to the practitioners reviewing the success stories of adopting automated testing, e.g., [54][55][56][57]  There is only one vote by a respondent for the "always-always" combination ("always" conducting both manual and automated testing).…”

Section: Figure 34-manual Versus Automated Testingmentioning

confidence: 99%

A survey of software engineering practices in Turkey

Garousi

Coşkunçay

Betin-Can

et al. 2015

Journal of Systems and Software

Self Cite

View full text Add to dashboard Cite

Section: Figure 34-manual Versus Automated Testingmentioning

confidence: 99%

A survey of software engineering practices in Turkey

Garousi

Coşkunçay

Betin-Can

et al. 2015

Journal of Systems and Software

Self Cite

View full text Add to dashboard Cite

“…As Figure 3 shows, in our research methodology, we also gathered inputs from our personal experience and qualitative evidence in the projects, the papers which have resulted from those projects (e.g., [52][53][54][55][56][57][58][59]), additional email and personal communication with the project staff at the time of working on this paper, and finally from the recent SLR study [3] about the set of challenges, patterns and anti-patterns that we chose as baselines.…”

Section: Research Methodology and Synthesis Methodsmentioning

confidence: 99%

Industry–academia collaborations in software testing: experience and success stories from Canada and Turkey

Garousi

Eskandar²,

Herkiloğlu

2016

Software Qual J

Self Cite

View full text Add to dashboard Cite

Collaboration between industry and academia supports improvement and innovation in industry and helps to ensure industrial relevance in academic research. However, many researchers and practitioners believe that the level of joint industry-academia collaborations (IAC) in software engineering (SE) is still relatively very low, compared to the amount of activity in each of the two communities. The goal of the empirical study reported in this paper is to characterize a set of collaborative industry-academia R&D projects in the area of software testing conducted by the authors (based in Canada and Turkey) with respect to a set of challenges, patterns and anti-patterns identified by a recent Systematic Literature Review (SLR) study, with the aim of contributing to the body of evidence in the area of IAC, for the benefit of SE researchers and practitioners in conducting successful IAC projects in software testing and in software engineering in general. To address the above goal, a pool of 10 IAC projects (6 completed, 2 failed and 2 ongoing) all in the area of software testing, which the authors have led or have had an active roles in, were selected as objects of study and were analyzed (both quantitatively and qualitatively) w.r.t. the set of selected challenges, patterns and anti-patterns. As outputs, the study presents a set of empirical findings and evidence-based recommendations, e.g.: it has been observed that even if an IAC project may seem perfect from many aspects, but one single major challenge (e.g., disagreement in confidentiality agreements) can lead to its failure. Thus, we recommend that both parties (academics and practitioners) consider all the challenges early on and proactively work together to eliminate the risk of challenges in IAC projects. We furthermore report correlation and interrelationship of challenges, patterns and anti-patterns with project success measures. This study hopes to encourage and benefit other SE researchers and practitioners in conducting successful IAC projects in software testing and in software engineering in general in the future.

show abstract

“…Further, Ramler and Wolfmaier (2006) analyze test automation from a cost-benefit perspective. Another class of related papers deals with decision-support to determine the optimal degree of automation adoption, such as a simulation model (Garousi and Pfahl 2015) or using a search-based approach (Amannejad et al 2014). Our work focuses on the industrial test case automation (scripting) order problem based on reuse; therefore, hereafter, we review works appearing in the literature dealing with (i) similarity-based approaches for test cases and (ii) test case prioritization.…”

Section: Previous Workmentioning

confidence: 99%

Similarity-based prioritization of test case automation

et al. 2018

View full text Add to dashboard Cite

The importance of efficient software testing procedures is driven by an ever increasing system complexity as well as global competition. In the particular case of manual test cases at the system integration level, where thousands of test cases may be executed before release, time must be well spent in order to test the system as completely and as efficiently as possible. Automating a subset of the manual test cases, i.e, translating the manual instructions to automatically executable code, is one way of decreasing the test effort. It is further common that test cases exhibit similarities, which can be exploited through reuse when automating a test suite. In this paper, we investigate the potential for reducing test effort by ordering the test cases before such automation, given that we can reuse already automated parts of test cases. In our analysis, we investigate several approaches for prioritization in a case study at a large Swedish vehicular manufacturer. The study analyzes the effects with respect to test effort, on four projects with a total of 3919 integration test cases constituting 35,180 test steps, written in natural language. The results show that for the four projects considered, the difference in expected manual effort between the best and the worst order found is on average 12 percentage points. The results also show that our proposed prioritization method is nearly as good as more resource demanding meta-heuristic approaches Software Qual J at a fraction of the computational time. Based on our results, we conclude that the order of automation is important when the set of test cases contain similar steps (instructions) that cannot be removed, but are possible to reuse. More precisely, the order is important with respect to how quickly the manual test execution effort decreases for a set of test cases that are being automated.

show abstract

A Search-Based Approach for Cost-Effective Software Test Automation Decision Support and an Industrial Case Study

Cited by 23 publications

References 19 publications

A survey of software engineering practices in Turkey

A survey of software engineering practices in Turkey

Industry–academia collaborations in software testing: experience and success stories from Canada and Turkey

Similarity-based prioritization of test case automation

Contact Info

Product

Resources

About