We propose Guided Random Testing (GRT), which uses static and dynamic analysis to include information on program types, data, and dependencies in various stages of automated test generation. Static analysis extracts knowledge from the system under test. Test coverage is further improved through state fuzzing and continuous coverage analysis. We evaluated GRT on 32 real-world projects and found that GRT outperforms major peer techniques in terms of code coverage (by 13 %) and mutation score (by 9 %). On the four studied benchmarks of Defects4J, which contain 224 real faults, GRT also shows better fault detection capability than peer techniques, finding 147 faults (66 %). Furthermore, in an in-depth evaluation on the latest versions of ten popular real-world projects, GRT successfully detects over 20 unknown defects that were confirmed by developers.

Abstract:Testing is one of the most resource-intensive activities in software development and consumes between 30 and 50% of total development costs according to many studies. Testing is however often not organized to maximize business value and not aligned with a project's mission. Path, branch, instruction, mutation, scenario, or requirement testing usually treat all aspects of software as equally important, while in practice 80% of the value often comes from 20% of the software. In order to maximize the return of investment gained from software testing, the management of testing needs to maximize its value contribution. In this chapter we motivate the need for value-based testing, describe practices supporting the management of value-based testing, outline a framework for value-based test management, and illustrate the framework with an example.

No abstract

Software Product Lines (SPLs) are families of related software systems distinguished by the set of features each one provides. Over the past decades SPLs have been the subject of extensive research and application both in academia and industry. SPLs practices have proven benefits such as better product customization and reduced time to market. Testing SPLs pose additional challenges stemming from the typically large number of product variants which make it infeasible to test every single one of them. In recent years, there has been an extensive research on applying Combinatorial Interaction Testing (CIT) for SPL testing. In this paper we present the first systematic mapping study on this subject. Our research questions aim to gather information regarding the techniques that have been applied, the nature of the case studies used for their evaluation, and what phases of CIT have been addressed. Our goal is to identify common trends, gaps, and opportunities for further research and application.

No abstract