A survey on software test automation return on investment, in organizations predominantly from Bengaluru, India

Abstract

Software industry has adopted automated testing widely. The most common method adopted is graphical user interface test automation for the functional scenarios to reduce manual testing and increase the repeatability. Reducing execution time and redundancy to achieve quality “go/no go decisions” provides rational for the executive management to allocate funds to adopt automation and invest in the setup including people, process, and tools to achieve faster time to market. There are a variety of practices engaged by testers, like frameworks, tools, methods, procedures, models, and technologies to achieve automation. Nonetheless, the actual effectiveness in terms of return on investment (ROI) is not known, though there are various formulas to calculate ROI of test automation. The factors that determine the ROI are maturity of test automation, purpose, or intent, picking right tests for automation, knowledge of the tester to derive test coverage, domain expertise, defining right metrics like defects found by automation runs, cost versus reduction of time, labor versus quality of scripts, repeatability, and results. These factors form the base of the questions designed for the survey. The paper presents a survey and analysis to understand the ROI of test automation from industry test professionals from both product and services organizations.

Development of a Maturity Model for Software Quality Assurance Practices

“…Also, a performance tool can be used that can run a large number of concurrent transactions that measure performance and compatibility. This comes in alignment with the finding of [12]. According to the authors, automation testing is challenging to achieve in organizations, with only 6% of professionals believing it is possible.…”

“…In conducting the survey, the author focused on the return on investment in automated testing, taking into account the following criteria: test coverage, test efficacy, test strategy, tools, cost of test automation, and people, which were distributed to professionals from various product and service organizations. According to the findings of this study, testing accounts for 28% of total IT spending for the last five years due to product and service management, as well as the ability to automate the appropriate test [12]. According to the study, unit testing with adequate automation should deliver technical richness that provides faster, cleaner, and higher quality outputs with very minimal maintenance costs.…”

“…Lastly, for some organizations, a lack of resources would consider quality assurance a luxury, and a preliminary level of quality practices would be sufficient. The authors of [12] support the fact that, in IT overall, there is uncertainty when it comes to quality assurance of the return of investments of quality factors such as investing in automation testing. As a result, there has not been enough literature and models investigating the SQA practices and models, both comprehensively and in a structured way.…”

“…The mix of the testing levels to reach an accuracy of the implementation and efficiency [12,51] Testing objective…”

“…This factor measures the degree of precision of the expected result of the applied testing activity to achieve the testing objective, such as acceptance testing, compatibility testing, execution time testing, penetration testing, quality of service testing, regression testing, robustness testing, safety testing, security testing, UI testing, usability testing. [12,52,53] Testing activity…”