A classification of product sampling for software product lines

Varshosaz, Mahsa; Al-Hajjaji, Mustafa; Thüm, Thomas; Runge, Tobias; Mousavi, Mohammad Reza; Schaefer, Ina

doi:10.1145/3233027.3233035

Cited by 63 publications

(57 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, we cannot take the whole sample of QuickSampler or UniGen and we rather need to take an excerpt of this sample. Various sub-sampling strategies can be considered [24], [40] either based on random, t-wise, etc. Without an uniform distribution, the sub-sampling process will operate over a non-representative configuration set, which may incidentally promote or underestimate some features.…”

Section: B Practical Implicationsmentioning

confidence: 99%

“…Configurable Systems Sampling: As outlined by a recent survey by Varshosaz et al [40], many artifacts of configurable systems have been considered for sampling, such as feature models, code, tests, etc. The authors also indicate that feature model remains the de facto standard input for many sampling techniques and note a certain prominence of greedy and metaheuristic-based algorithms, which are heavy consumers of random samples.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Plazar

Acher

Perrouin

et al. 2019

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

Uniform or near-uniform generation of solutions for large satisfiability formulas is a problem of theoretical and practical interest for the testing community. Recent works proposed two algorithms (namely UniGen and QuickSampler) for reaching a good compromise between execution time and uniformity guarantees, with empirical evidence on SAT benchmarks. In the context of highly-configurable software systems (e.g., Linux), it is unclear whether UniGen and QuickSampler can scale and sample uniform software configurations. In this paper, we perform a thorough experiment on 128 real-world feature models. We find that UniGen is unable to produce SAT solutions out of such feature models. Furthermore, we show that QuickSampler does not generate uniform samples and that some features are either never part of the sample or too frequently present. Finally, using a case study, we characterize the impacts of these results on the ability to find bugs in a configurable system. Overall, our results suggest that we are not there: more research is needed to explore the cost-effectiveness of uniform sampling when testing large configurable systems.

show abstract

Section: B Practical Implicationsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Plazar

Acher

Perrouin

et al. 2019

2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST)

View full text Add to dashboard Cite

show abstract

“…An FM is treelike structure encompassing a set of features and constraints among them. Since feature modeling is a popular way of documenting commonality and variability of an SPL in both academic and industrial communities [4], [10], [64], it makes sense to develop FM-based SPL testing techniques. clause if the feature model is transformed to a Boolean formula in conjunctive normal form (CNF) [65].…”

Section: Feature Model Configurations As Test Casesmentioning

confidence: 99%

“…Moreover, even if one could test all valid products (for small-scale FMs), it is likely to be inefficient because software testers often have a limited amount of test resources [8], like time and budgets. Therefore, software testers are seeking sampling approaches to reduce the size of products under test so as to meet the release deadline or resource constraints [10]. The combinatorial interaction testing (CIT) [11], among others, is a prominent approach that has been introduced to reduce the size of the test suites while achieving a certain coverage of feature combinations [12], [13].…”

Section: Introductionmentioning

confidence: 99%

Looking For Novelty in Search-based Software Product Line Testing

Xiang¹

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Intuitively, the generation method should allow the formulation of architectures with diverse shapes (in terms of layers). Such Manuscript submitted to ACM techniques have been intensively studied and compared in the area of configurable systems [22,35]. Here, we aim at demonstrating that it is possible to represent and search the DNN architecture space so that we can find useful configurations.…”

Section: Forming Architecturesmentioning

confidence: 99%

Automated Search for Configurations of Convolutional Neural Network Architectures

Ghamizi

Cordy

Papadakis

et al. 2019

Proceedings of the 23rd International Systems and Software Product Line Conference - Volume A

View full text Add to dashboard Cite

embourg Deep Neural Networks (DNNs) are intensively used to solve a wide variety of complex problems. Although powerful, such systems require manual configuration and tuning. To this end, we view DNNs as configurable systems and propose an end-to-end framework that allows the configuration, evaluation and automated search for DNN architectures. Therefore, our contribution is threefold. First, we model the variability of DNN architectures with a Feature Model (FM) that generalizes over existing architectures. Each valid configuration of the FM corresponds to a valid DNN model that can be built and trained. Second, we implement, on top of Tensorflow, an automated procedure to deploy, train and evaluate the performance of a configured model. Third, we propose a method to search for configurations and demonstrate that it leads to good DNN models. We evaluate our method by applying it on image classification tasks (MNIST, CIFAR-10) and show that, with limited amount of computation and training, our method can identify high-performing architectures (with high accuracy). We also demonstrate that we outperform existing state-of-the-art architectures handcrafted by ML researchers. Our FM and framework have been released to support replication and future research.

show abstract

A classification of product sampling for software product lines

Cited by 63 publications

References 74 publications

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?

Looking For Novelty in Search-based Software Product Line Testing

Automated Search for Configurations of Convolutional Neural Network Architectures

Contact Info

Product

Resources

About