Uniform and scalable sampling of highly configurable systems

Heradio, Rubén; Fernández-Amorós, David; Galindo, José; Benavides, David; Batory, Don

doi:10.1007/s10664-021-10102-5

Cited by 22 publications

(15 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They participate in non-regression testing. However, SPL testing [15] and ML testing [55] are inherently difficult activities that we do not yet address; Many algorithms built into SPL are too resource-intensive (CPU, memory, and time) to consider sampling techniques [23]. Nonetheless, we believe that some work on SPL configurations opens up new opportunities to help build portfolios for automatic algorithm selection [29].…”

Section: Discussionmentioning

confidence: 99%

“…Then the main issue is not to determine the configuration workflow that best suits the actors according to the previous configurations [45], but to guide them in composing a solution for an unprecedentedly studied problem. Concurrently, it is not a question of generating random samples [23], whose relevance could not be precisely verified (e.g., stuffing the SPL with all the available algorithms and pre-processing components from the literature). Instead, it is more a matter of enriching our knowledge by systematically studying new validated configurations.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Evolvable SPL management with partial knowledge

Amraoui

Blay–Fornarino

Collet

et al. 2022

Proceedings of the 26th ACM International Systems and Software Product Line Conference - Volume A

View full text Add to dashboard Cite

In Machine Learning (ML), the resolution of anomaly detection problems in time series presents a great diversity of practices as it can correspond to many different contexts. These practices cover both grasping the business problem and designing the solution itself. By practice, we designate explicit and implicit steps toward resolving a problem, while a solution corresponds to a combination of algorithms selected for their performance on a given problem. Two related issues arise. The first one is that the practices are individual and not explicitly mutualized. The second one is that choosing one solution over another is all the more difficult to justify because the space of solutions and the evaluation criteria are vast and evolve rapidly with the advances in ML. To solve these issues and tame the evolving diversity in ML, a Software Product Line (SPL) approach can be envisaged to represent the variable set of solutions. However, this requires characterizing an ML business problem through an explicit set of criteria and justifying one ML solution over all others. The resolution of anomaly detection problems is thus different from finding the best configuration workflow from past configurations but lies more in guiding the configuration towards a solution that may never have been studied before. This paper proposes an SPL approach that capitalizes on past practices by exploiting a variability-aware representation to detect new criteria and constraints when practices adopt different solutions to seemingly similar problems. We report on the evaluation of our approach using a set of applications from the literature and an ML software company. We show how the analysis of practices makes it possible to consolidate the knowledge contained in the SPL.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Evolvable SPL management with partial knowledge

Amraoui

Blay–Fornarino

Collet

et al. 2022

Proceedings of the 26th ACM International Systems and Software Product Line Conference - Volume A

View full text Add to dashboard Cite

show abstract

“…INFORMEDQX Runtime Performance. We have evaluated the performance of INFORMEDQX compared to QUICKXPLAIN on the basis of the Linux-2.6.33.3 configuration knowledge base taken from Diverso Lab's benchmark 1 (Heradio et al 2022). The characteristics of this knowledge base are the following: #features = 6,467; #relationships = 6,322; and #cross-tree constraints = 7,650.…”

Section: Analysis Of Informedqxmentioning

confidence: 99%

INFORMEDQX: Informed Conflict Detection for Over-Constrained Problems

Le,

Felfernig,

Tran

et al. 2024

AAAI

View full text Add to dashboard Cite

Conflict detection is relevant in various application scenarios, ranging from interactive decision-making to the diagnosis of faulty knowledge bases. Conflicts can be regarded as sets of constraints that cause an inconsistency. In many scenarios (e.g., constraint-based configuration), conflicts are repeatedly determined for the same or similar sets of constraints. This misses out on the valuable opportunity for leveraging knowledge reuse and related potential performance improvements, which are extremely important, specifically interactive constraint-based applications. In this paper, we show how to integrate knowledge reuse concepts into non-instructive conflict detection. We introduce the InformedQX algorithm, which is a reuse-aware variant of QuickXPlain. The results of a related performance analysis with the Linux-2.6.3.33 configuration knowledge base show significant improvements in terms of runtime performance compared to QuickXPlain.

show abstract

“…Being strictly more succinct means that some exponential space savings can be achieved by targeting the d-DNNF language instead of the OBDD one. These properties of d-DNNF circuits, the various sets of tractable queries supported by d-DNNF circuits and the existence of "efficient" compilers, explain why the d-DNNF language, developed two decades ago for some AI purposes (especially, model-based diagnosis) has been spreading over a number of domains that go beyond AI; in particular, theoretical computer science, database theory, and more recently software engineering [47,48].…”

Section: D-dnnf-based Reasoning For Feature Modelsmentioning

confidence: 99%

Reasoning on Feature Models: Compilation-Based vs. Direct Approaches

Bourhis¹,

Duchien²,

Dusart³

et al. 2023

Preprint

View full text Add to dashboard Cite

Analyzing a Feature Model (FM) and reasoning on the corresponding configuration space is a central task in Software Product Line (SPL) engineering. Problems such as deciding the satisfiability of the FM and eliminating inconsistent parts of the FM have been well resolved by translating the FM into a conjunctive normal form (CNF) formula, and then feeding the CNF to a SAT solver. However, this approach has some limits for other important reasoning issues about the FM, such as counting or enumerating configurations. Two mainstream approaches have been investigated in this direction: (i) direct approaches, using tools based on the CNF representation of the FM at hand, or (ii) compilation-based approaches, where the CNF representation of the FM has first been translated into another representation for which the reasoning queries are easier to address. Our contribution is twofold. First, we evaluate how both approaches compare when dealing with common reasoning operations on FM, namely counting configurations, pointing out one or several configurations, sampling configurations, and finding optimal configurations regarding a utility function. Our experimental results show that the compilation-based is efficient enough to possibly compete with the direct approaches and that the cost of translation (i.e., the compilation time) can be balanced when addressing sufficiently many complex reasoning operations on large configuration spaces. Second, we provide a Java-based automated reasoner that supports these operations for both approaches, thus eliminating the burden of selecting the appropriate tool and approach depending on the operation one wants to perform.

show abstract

Uniform and scalable sampling of highly configurable systems

Cited by 22 publications

References 53 publications

Evolvable SPL management with partial knowledge

Evolvable SPL management with partial knowledge

INFORMEDQX: Informed Conflict Detection for Over-Constrained Problems

Reasoning on Feature Models: Compilation-Based vs. Direct Approaches

Contact Info

Product

Resources

About