Economic perspectives in test automation

Abstract

Testing is a major cost factor in software development. Test automation has been proposed as one solution to reduce these costs. Test automation tools promise to increase the number of tests they run and the frequency at which they run them. So why not automate every test? In this paper we discuss the question "When should a test be automated?" and the trade-off between automated and manual testing. We reveal problems in the overly simplistic cost models commonly used to make decisions about automating testing. We introduce an alternative model based on opportunity cost and present influencing factors on the decision of whether or not to invest in test automation. Our aim is to stimulate discussion about these factors as well as their influence on the benefits and costs of automated testing in order to support researchers and practitioners reflecting on proposed automation approaches.

Automated Testing of Mobile Applications Using a Robotic Arm

“…The first method is testing an application on an emulator or on a device by manually testing all functionalities. This can be cheaper than other test methods if done in the beginning of a software project [12]. To use this method the software should be given to a person not involved in developing the application.…”

Trade-off between automated and manual software testing

“…For example, Ramler and Wolfmaier (2006) summarize the differences between manual and automated testing by suggesting that automation should be used to prevent further errors, while manual testing is better suited for finding new and unexpected errors. According to Dustin et al (1999), reasons for using automated software testing are, for example, that manual testing is time consuming, and that test automation increases efficiency, especially in regression testing, where test cases are executed iteratively after making changes to the software.…”

“…Torkar and Mankefors (2003) found that 60% of the developers in their survey claimed that verification and validation were the first to be neglected in cases of resource shortages during a project. It is a common view that if some of the test process phases could be automated and the existing processes themselves streamlined, the available resources could be directed towards additional testing or gaining savings (Ramler and Wolfmaier 2006). Berner et al (2005) estimate that most of the test cases in one project are run at least five times, and one-fourth over 20 times.…”

“…Persson and Yilmaztürk (2004) note that the decision on what to automate and what to test manually should be defined early in the project to avoid problems. To support decision making, Ramler and Wolfmaier (2006) have developed a model for estimating test automation costs.…”

How to Generate and Import Functional Test Cases into a Project Management Software System using Natural Language Processing

“…Humans still needed to design, build, and maintain automation scripts, but after built they can be deployed automatically. To fulfill the software quality, manual and automation approaches are required [3]. The motivation for this research is to save time and improve the quality of testing by providing an additional approach to generate test cases for software testing.…”