A first systematic mapping study on combinatorial interaction testing for software product lines

Abstract

Software Product Lines (SPLs) are families of related software systems distinguished by the set of features each one provides. Over the past decades SPLs have been the subject of extensive research and application both in academia and industry. SPLs practices have proven benefits such as better product customization and reduced time to market. Testing SPLs pose additional challenges stemming from the typically large number of product variants which make it infeasible to test every single one of them. In recent years, there has been an extensive research on applying Combinatorial Interaction Testing (CIT) for SPL testing. In this paper we present the first systematic mapping study on this subject. Our research questions aim to gather information regarding the techniques that have been applied, the nature of the case studies used for their evaluation, and what phases of CIT have been addressed. Our goal is to identify common trends, gaps, and opportunities for further research and application.

Multi-objective test case prioritization in highly configurable systems: A case study

“…Since typical HCSs can have thousands or even millions of different configurations, several sampling techniques have been proposed to reduce the number of configurations to be tested (e.g. [46,50,55]). Salient among them is pairwise testing whose goal is to select test suites that contain all possible combinations of pairs of features [46].…”

“…[46,50,55]). Salient among them is pairwise testing whose goal is to select test suites that contain all possible combinations of pairs of features [46]. As an example, Table 3 shows the set of configurations obtained when applying pairwise testing to the model in Figure 1.…”

“…feature combinations, as representative exemplars on which to perform the testing tasks. Recently, we performed a systematic mapping study to delve into more detail on the subject [46]. This mapping study identified over forty different approaches that rely on diverse techniques, such as genetic and greedy algorithms, that were evaluated also with multiple problem domains of different characteristics.…”

Combinatorial Interaction Testing of Software Product Lines: A Mapping Study

“…Recently, a mapping study published by Lopez-Herrejon et al (2015) reported that numerous works have been done on Combinatorial Interaction Testing in SPL, by looking from two perspectives which is what phase and how phase of CIT. Similar to our work, they also presented state-of-the-art works based on the techniques employed in CIT.…”

A Univariate Marginal Approach for Pairwise Testing of Software Product Lines

“…A number of researchers had proposed a couple of prominent strategies to reduce the combinatorial explosion of feature configuration testing [6]. Based on our literature, most of the current approaches are based on greedy algorithms with only a few works leveraged the potential of search-based techniques, which have been widely used in the single software development testing.…”