Combinatorial interaction testing of tangled configuration options

Javeed, Arsalan; Yılmaz, Cemal

doi:10.1109/icstw.2015.7107438

Cited by 5 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, covering all valid 1-combinations (i.e., all virtual settings) guarantees to obtain full coverage under DC. Consequently, 1way structure-based F-CIT objects are the same/similar combinatorial objects we introduced in our short paper [12], but expressed in F-CIT, demonstrating the expressiveness of F-CIT.…”

Section: Coverage Criterionmentioning

confidence: 56%

“…Then, for each testable entity e ∈ E, we check to see if e has already been covered by a test case t ∈ T (lines 4-9), i.e., if there exists a test case t ∈ T , which is consistent with e (line 5). If no such test case is found, a new F-CIT test case covering e, is generated by solving the constraint e ∧ C and T is populated with the newly generated test case (lines [10][11][12]. Once all the testable entities in E have been processed, T is returned as the F-CIT object computed (line 14).…”

Section: Algorithmmentioning

confidence: 99%

“…In [26], [12], we introduced a novel CIT object, which given a structural coverage criterion, such as decision coverage (DC), computes a "minimal" test suite to obtain full coverage under the criterion. In this work, we not only express the same coverage criterion using F-CIT, demonstrating the expressiveness of F-CIT, but also generalize the aforementioned coverage criterion to higher coverage strengths, demonstrating the flexibility of F-CIT.…”

Section: Coverage Criterionmentioning

confidence: 99%

“…For example, if the coverage criterion in our running example was changed from t-way coverage to decision coverage [12], where the goal is to cover every outcome of a decision in Figure 1a at least once, then, to guarantee full coverage, the strength of the standard covering array to be used would be at least 4 (i.e., t ≥ 4). This is because the outcome of the decision in line 2 (Figure 1a) depends on the interactions among 4 options, namely o 1 , o 2 , o 3 , and o 4 .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Flexible Combinatorial Interaction Testing

Mercan

Javeed

Yılmaz

2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

We present Flexible Combinatorial Interaction Testing (F-CIT), which aims to improve the flexibility of combinatorial interaction testing (CIT) by eliminating the necessity of developing specialized constructors for CIT problems that cannot be efficiently and effectively addressed by the existing CIT constructors. F-CIT expresses the entities to be covered and the space of valid test cases, from which the samples are drawn to obtain full coverage, as constraints. Computing an F-CIT object (i.e., a set of test cases obtaining full coverage under a given coverage criterion) then turns into an interesting constraint solving problem, which we call cov-CSP. cov-CSP aims to divide the constraints, each representing an entity to be covered, into a minimum number of satisfiable clusters, such that a solution for a cluster represents a test case and the collection of all the test cases generated (one per cluster) constitutes an F-CIT object, covering each required entity at least once. To solve the cov-CSP problem, thus to compute F-CIT objects, we first present two constructors. One of these constructors attempts to cover as many entities as possible in a cluster before generating a test case, whereas the other constructor generates a test case first and then marks all the entities accommodated by this test case as covered. We then use these constructors to evaluate F-CIT in three studies, each of which addresses a different CIT problem. In the first study, we develop structure-based F-CIT objects to obtain decision coverage-adequate test suites. In the second study, we develop order-based F-CIT objects, which enhance a number of existing order-based coverage criteria by taking the reachability constraints imposed by graph-based models directly into account when computing interaction test suites. In the third study, we develop usage-based F-CIT objects to address the scenarios, in which standard covering arrays are not desirable due to their sizes, by choosing the entities to be covered based on their usage statistics collected from the field. We also carry out user studies to further evaluate F-CIT. The results of these studies suggest that F-CIT is more flexible than the existing CIT approaches.

show abstract

Section: Coverage Criterionmentioning

confidence: 56%

Section: Algorithmmentioning

confidence: 99%

Section: Coverage Criterionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Flexible Combinatorial Interaction Testing

Mercan

Javeed

Yılmaz

2022

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Nevertheless, verification and validation on software system are needed to ensure the correctness and high quality of product value before delivering. Due to budget and time constraints, exhaustive testing in such a modern software system is practically impossible as it requires extensive efforts and excessive resources [1,2].…”

Section: Introductionmentioning

confidence: 99%

Input Modeling Prioritization Using Statistically User Profile for Pairwise Test Case Generation with Constraints Handling

Nakornburi¹,

Suwannasart²

2017

View full text Add to dashboard Cite

Abstract. Pairwise testing is a widely used technique for software testing with the reduced size of the test suite and able to detect interactions that trigger the system's faults. In addition, pairwise testing test suites must be able to take constraints between input parameters and parameter values into account. In current practice, identifying and selecting input parameters and parameter values usually depends on tester skills that might not be sufficient. Input parameters and parameter values modeling and tools for easily guiding and prioritizing the selection of optimal input parameters and parameter values for the SUT is also required. In this work, we present an approach for prioritizing input parameters and parameter values modeling using statistical user profile. Our approach is implemented in a tool called UPPTCT which provides the ability to handle constraints on input parameters and parameter values for pairwise testing in order to generate test cases. We conduct experiments to evaluate test case effectiveness and compare our tool with other renowned pairwise test generation and constraints handling tools. The experimental results show that the effectiveness of our approach is significantly more efficient and effective than random testing as a large portion of reported defects with regard to statically user profile were caught by our approach. Furthermore, our tool performs better in some cases and performs comparable results for generating test cases upon input parameters and parameter values for both with constraints handling and without constraints handling.

show abstract

A tool for constrained pairwise test case generation using statistical user profile based prioritization

Nakornburi

Suwannasart

2016

2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE)

View full text Add to dashboard Cite

Combinatorial interaction testing of tangled configuration options

Cited by 5 publications

References 14 publications

Flexible Combinatorial Interaction Testing

Flexible Combinatorial Interaction Testing

Input Modeling Prioritization Using Statistically User Profile for Pairwise Test Case Generation with Constraints Handling

A tool for constrained pairwise test case generation using statistical user profile based prioritization

Contact Info

Product

Resources

About