Towards Sampling and Simulation-Based Analysis of Featured Weighted Automata

Cordy, Maxime; Legay, Axel; Lazreg, Sami; Collet, Philippe

doi:10.1109/formalise.2019.00015

Cited by 3 publications

(2 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some papers consider the problem of sampling uniformly variants in order to study their characteristics (e.g. performance [KGS + 19] and other quality requirements [CLLC19]) and infers the characteristics of the other variants. Others attempt to compute a sample that covers all combinations of VIS variation points [PTR +…”

Section: Sampling Of Vismentioning

confidence: 99%

Statistical model checking for variability-intensive systems: applications to bug detection and minimization

et al. 2021

Self Cite

View full text Add to dashboard Cite

We propose a new Statistical Model Checking (SMC) method to identify bugs in variability-intensive systems (VIS). The state-space of such systems is exponential in the number of variants, which makes the verification problem harder than for classical systems. To reduce verification time, we propose to combine SMC with featured transition systems (FTS)—a model that represents jointly the state spaces of all variants. Our new methods allow the sampling of executions from one or more (potentially all) variants. We investigate their utility in two complementary use cases. The first case considers the problem of finding all variants that violate a given property expressed in Linear-Time Logic (LTL) within a given simulation budget. To achieve this, we perform random walks in the featured transition system seeking accepting lassos. We show that our method allows us to find bugs much faster (up to 16 times according to our experiments) than exhaustive methods. As any simulation-based approach, however, the risk of Type-1 error exists. We provide a lower bound and an upper bound for the number of simulations to perform to achieve the desired level of confidence. Our empirical study involving 59 properties over three case studies reveals that our method manages to discover all variants violating 41 of the properties. This indicates that SMC can act as a coarse-grained analysis method to quickly identify the set of buggy variants. The second case complements the first one. In case the coarse-grained analysis reveals that no variant can guarantee to satisfy an intended property in all their executions, one should identify the variant that minimizes the probability of violating this property. Thus, we propose a fine-grained SMC method that quickly identifies promising variants and accurately estimates their violation probability. We evaluate different selection strategies and reveal that a genetic algorithm combined with elitist selection yields the best results.

show abstract

Section: Sampling Of Vismentioning

confidence: 99%

Statistical model checking for variability-intensive systems: applications to bug detection and minimization

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such work considers the problem of sampling uniformly variants in order to study their characteristics (e.g. performance [22] and other quality requirements [15]) and infers those of the other variants. Recently, Thüm et al [35] survey different strategies for the performance analysis of VIS, including the sampling of variants and family-based test generation, which is based on the same idea of executing test cases common to multiple variants.…”

Section: Other Related Workmentioning

confidence: 99%

Statistical Model Checking for Variability-Intensive Systems

Cordy

Papadakis

Legay

2020

Fundamental Approaches to Software Engineering

Self Cite

View full text Add to dashboard Cite

We propose a new Statistical Model Checking (SMC) method to discover bugs in variability-intensive systems (VIS). The state-space of such systems is exponential in the number of variants, which makes the verification problem harder than for classical systems. To reduce verification time, we sample executions from a featured transition systema model that represents jointly the state spaces of all variants. The combination of this compact representation and the inherent efficiency of SMC allows us to find bugs much faster (up to 16 times according to our experiments) than other methods. As any simulation-based approach, however, the risk of Type-1 error exists. We provide a lower bound and an upper bound for the number of simulations to perform to achieve the desired level of confidence. Our empirical study involving 59 properties over three case studies reveals that our method manages to discover all variants violating 41 of the properties. This indicates that SMC can act as a low-cost-high-reward method for verifying VIS.

show abstract