Mutation-Based Web Test Case Generation

Abstract

One way to increase software quality is testing. Generating test cases with a high level of coverage may be neither a simple nor fast task and when the software suffers changes, it is even more difficult to guarantee this coverage. Mutation testing is a fault-based software testing technique that introduces small faults in the code -called mutations -and evaluates if the test suite is able to detect these faults, i.e., distinguish the results obtained in the original version of the program from the results obtained in the mutated version, based on a mutation score. It is measured by 'killing' or not the mutants when the tests are executed: if the mutant is killed, then the test case is able to detect this fault; otherwise, if the mutant survived, it is necessary to add more test cases to detect and cover this fault. In this project, this technique is applied in a different way: generate test cases.This project aims to use information about the most frequent paths of the user interaction on a software service, collected by a web analytics tool in order to generate test cases and extend them with mutations (mutations applied over the tests) that can make sense in web testing. There are some approaches which generate automatically test cases. Nevertheless, there are several challenges to overcome in this area such as the complexity of the regression test, the necessity of knowing about this kind of test and how to implement them. It is also necessary to have documented all the input and output data and the steps to apply the test case and some tools actually cannot capture input values and it implies the needing of insert it manually by the tester.This work includes the definition of a catalog of mutations such as, for example, change the order of execution of the tasks. The main objective is defining the appropriate mutations and implement them automatically in the test cases to check if they have different behaviour from the original test cases. This approach will allow to automatically generate test cases based on the logs information and web usage data of services and extend them with some possible mutations. The implementation of the generator of the test cases is also included. In this way, it is expected that the quality of the test suite will be improved once there will be test cases that simulate wrong user behaviors and aspects that are not tested yet. Consequently, the coverage will be increased with the new test cases. This tool is useful to detect automatically errors in web pages and have a better test suite based on the most frequent paths of the service.This approach is validated with a simple developed scenario and with real usage information from the Polytechnic Institute of Viana do Castelo website. A considerable number of mutated test cases with different behaviour were generated in both situations.

On Understanding Contextual Changes of Failures

“…Generating test cases through mutations [36] has been applied in web page testing [37], and tools like Sapienz [38] are already following this approach and successfully detecting bugs in mobile applications used by millions of people.…”