Estimating Return on Investment for GUI Test Automation Frameworks

Dobslaw, Felix; Feldt, Robert; Michaelsson, David; Haar, Patrik; Neto, Francisco Gomes de Oliveira; Torkar, Richard

doi:10.1109/issre.2019.00035

Cited by 14 publications

(5 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…104,105 Automation ROI estimation helps to justify the time and material spent on the automation suite with respect to quality. [106][107][108][109][110]…”

Section: Review Of Literaturementioning

confidence: 99%

A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

Reanzi

Thangaiah

2021

International Journal of Engineering Business Management

View full text Add to dashboard Cite

Software industry has adopted automated testing widely. The most common method adopted is graphical user interface test automation for the functional scenarios to reduce manual testing and increase the repeatability. Reducing execution time and redundancy to achieve quality “go/no go decisions” provides rational for the executive management to allocate funds to adopt automation and invest in the setup including people, process, and tools to achieve faster time to market. There are a variety of practices engaged by testers, like frameworks, tools, methods, procedures, models, and technologies to achieve automation. Nonetheless, the actual effectiveness in terms of return on investment (ROI) is not known, though there are various formulas to calculate ROI of test automation. The factors that determine the ROI are maturity of test automation, purpose, or intent, picking right tests for automation, knowledge of the tester to derive test coverage, domain expertise, defining right metrics like defects found by automation runs, cost versus reduction of time, labor versus quality of scripts, repeatability, and results. These factors form the base of the questions designed for the survey. The paper presents a survey and analysis to understand the ROI of test automation from industry test professionals from both product and services organizations.

show abstract

“…104,105 Automation ROI estimation helps to justify the time and material spent on the automation suite with respect to quality. [106][107][108][109][110]…”

Section: Review Of Literaturementioning

confidence: 99%

A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

Reanzi

Thangaiah

2021

International Journal of Engineering Business Management

View full text Add to dashboard Cite

show abstract

“…Given the following candidate web elements (|| means that an attribute can have multiple values): {widget_id:"202",tag:"a",text:"Beauty, Health",href:"https://www.aliexpress.com/category/66/health-beauty.html",location :"277,541",area:"1584",shape:"488",is_button:"no",xpath:"/html/body/div/div [5]/div/div [2]/div/div [2]/div/div [2]/dl [11]/dt/span/a",neighbor_text:"toys kids & babies outdoor fun sports beauty health hair automobiles motorcycles home improvement tools"} {widget_id:"200",tag:"span || a",text:"Outdoor Fun & Sports",href:"https://www.aliexpress.com/category/18/sportsentertainment.html",location:"236,497",area:"8400",shape:"685",is_button:"no",xpath:"/html/body/div/div [5]/div/div [2]/div/div [2]/div/div [2]/dl [10]/dt/span",neighbor_text:"bags & shoes toys kids babies outdoor fun sports beauty health hair automobiles motorcycles"} {widget_id:"201",tag:"span",text:"Beauty, Health & Hair",location:"236,532",area:"8400",shape:"685",is_button:"no",xpath :"/html/body/div/div [5]/div/div [2]/div/div [2]/div/div [2]/dl [11]/dt/span",neighbor_text:"toys kids & babies outdoor fun sports beauty health hair automobiles motorcycles home improvement tools"} {widget_id:"204",tag:"span || a",text:"Automobiles & Motorcycles",href:"https://www.aliexpress.com/category/34/ automobiles-motorcycles.html",location:"236,567",area:"8400",shape:"685",is_button:"no",xpath:"/html/body/div/div [5]/div/div [2]/div/div [2]/div/div [2]/dl [12]/dt/span",neighbor_text:"outdoor fun & sports beauty health hair automobiles motorcycles home improvement tools"} {widget_id:"197",tag:"span",text:"Toys , Kids & Babies",location:"236,462",area:"8400",shape:"685",is_button:"no",xpath :"/html/body/div/div [5]/div/div [2]/div/div [2]/div/div [2]/dl [9]/dt/span",neighbor_text:"home pet & appliances home bags shoes toys kids babies outdoor fun sports beauty health hair"} {widget_id:"199",tag:"a",text:"Kids & Babies",href:"https://www.aliexpress.com/category/1501/mother-kids.html",location :"313,471",area:"1476",shape:"455",is_button:"no",xpath:"/html/body/div/div [5]/div/div [2]/div/div…”

Section: E Prompt Engineeringmentioning

confidence: 99%

“…However, it is common for new software releases to introduce changes that can break existing automated regression tests, which require maintenance efforts and costs to update and repair the test scripts. The maintenance cost is exceptionally high when testing an application through its GUI, as GUIs frequently change between releases [8], [9], [10]. In addition, GUI scripts are subject to breaking from changes to the underlying logic and architecture of the AUT that modifies its behavior.…”

Section: Introductionmentioning

confidence: 99%

Similarity-based Web Element Localization for Robust Test Automation

Nass

Alégroth

Feldt

et al. 2023

ACM Trans. Softw. Eng. Methodol.

Self Cite

View full text Add to dashboard Cite

Non-robust (fragile) test execution is a commonly reported challenge in GUI-based test automation, despite much research and several proposed solutions. A test script needs to be resilient to (minor) changes in the tested application but, at the same time, fail when detecting potential issues that require investigation. Test script fragility is a multi-faceted problem. However, one crucial challenge is how to reliably identify and locate the correct target web elements when the website evolves between releases or otherwise fail and report an issue. This paper proposes and evaluates a novel approach called similarity-based web element localization (Similo), which leverages information from multiple web element locator parameters to identify a target element using a weighted similarity score. This experimental study compares Similo to a baseline approach for web element localization. To get an extensive empirical basis, we target 48 of the most popular websites on the Internet in our evaluation. Robustness is considered by counting the number of web elements found in a recent website version compared to how many of these existed in an older version. Results of the experiment show that Similo outperforms the baseline; it failed to locate the correct target web element in 91 out of 801 considered cases (i.e., 11%) compared to 214 failed cases (i.e., 27%) for the baseline approach. The time efficiency of Similo was also considered, where the average time to locate a web element was determined to be four milliseconds. However, since the cost of web interactions (e.g., a click) is typically on the order of hundreds of milliseconds, the additional computational demands of Similo can be considered negligible. This study presents evidence that quantifying the similarity between multiple attributes of web elements when trying to locate them, as in our proposed Similo approach, is beneficial. With acceptable efficiency, Similo gives significantly higher effectiveness (i.e., robustness) than the baseline web element localization approach.

show abstract

“…Self-healing mechanisms for web-based applications have been developed with the intention of autonomously identifying and resolving problems without the need for human intervention. (Dobslaw et al, 2019) Although there exists a range of tools and frameworks for this particular objective, they typically belong to the domain of Automated Testing and Monitoring tools. Presented below are a few examples of tools equipped with the ability to self-heal.…”

Section: Self-healing Supported Toolsmentioning

confidence: 99%

Effortless Test Maintenance: A Critical Review of Self-Healing Frameworks

Khankhoje

2023

IJRASET

View full text Add to dashboard Cite

As the intricacy of web applications continues to escalate, the difficulties linked with the upkeep of testing in web-based automation have grown more apparent. Conventional automation scripts frequently encounter difficulties in adjusting to frequent modifications in web components, resulting in substantial manual endeavours and diminished testing efficacy. In reaction to these challenges, self-correcting web-based automation frameworks have emerged as a promising resolution. This critical review paper offers a comprehensive examination of web-based automation frameworks that have the ability to selfheal. It provides a thorough analysis of their principles, mechanisms, and real-world applications. Through the categorization and analysis of these frameworks' essential components, this review aims to elucidate their effectiveness in addressing the continuously evolving nature of web applications.While acknowledging the advancements made possible by self-healing frameworks, this review also explores the inherent challenges and limitations they possess. It outlines potential areas for future research and highlights emerging trends, such as the integration of artificial intelligence and machine learning, which have the potential to further enhance the self-healing capabilities of these frameworks. In summary, this evaluative critique offers a valuable asset for researchers, practitioners, and organisations aiming to optimize the process of managing tests in web-based automation. Through comprehending the fundamental concepts and potential compromises of self-healing frameworks for web-based automation, testing teams can strive towards effortless test maintenance and enhanced quality of web applications

show abstract

Estimating Return on Investment for GUI Test Automation Frameworks

Cited by 14 publications

References 33 publications

A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

Similarity-based Web Element Localization for Robust Test Automation

Effortless Test Maintenance: A Critical Review of Self-Healing Frameworks

Contact Info

Product

Resources

About