Dependence analysis in reduction of requirement based test suites

Vaysburg, B.; Tahat, Luay; Korel, Bogdan

doi:10.1145/566172.566188

Cited by 35 publications

(22 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other work has focused on model-based test suite minimization [32,33]. Vaysburg et al introduced a minimization technique for model-based test suites that uses dependence analysis of Extended Finite State Machines (EFSMs) [32]. Each test case for the model is a sequence of transitions.…”

Section: Testing Requirementsmentioning

confidence: 99%

Regression testing minimization, selection and prioritization: a survey

Yoo¹,

Harman²

2012

Softw. Test. Verif. Reliab.

1,058

594

View full text Add to dashboard Cite

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test suites. A number of different approaches have been studied to maximize the value of the accrued test suite: minimization, selection and prioritization. Test suite minimization seeks to eliminate redundant test cases in order to reduce the number of tests to run. Test case selection seeks to identify the test cases that are relevant to some set of recent changes. Test case prioritization seeks to order test cases in such a way that early fault detection is maximized. This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.This survey aims to collect and consider papers that deal with three regression testing techniques: test suite minimization, RTS and test case prioritization. Our intention is not to undertake a systematic review, but rather to provide a broad state-of-the-art view on these related fields. Many different approaches have been proposed to aid regression testing, which has resulted in a body of literature that is spread over a wide variety of domains and publication venues. The majority of surveyed literature has been published in the software engineering domain, and especially in the software testing and software maintenance literature. However, the regression testing literature also overlaps with those of programming language analysis, empirical software engineering and software metrics.Therefore, the paper selection criteria on which this survey is based are the problems considered in papers, while focusing on the specific topics of minimization, selection and prioritization. The formal definitions of these problems are presented in Section 2.2. The selected papers are listed in the Appendix. Fast abstracts and short papers have been excluded. BACKGROUNDThis section introduces the basic concepts and definitions that form a nomenclature of regression testing and minimization, selection and prioritization techniques. Definition 1 (Test Suite Minimization Problem)Given: A test suite, T , a set of test requirements {r 1 , . . .,r n }, that must be satisfied to provide the desired 'adequate' testing of the program, and subsets of T , T 1 , . . ., T n , one associated with each of the r i s such that any one of the test cases t j belonging to T i can be used to achieve requirement r i .Problem: Find a representative set, T , of test cases from T that satisfies all r i s.The testing criterion is satisfied when every test requirement in {r 1 , . . .,r n } is satisfied. A test requirement, r i , is satisfied by any test case, t j , that belongs to the T i , a subset of T . Therefore, the representative set of test cases is the hitting set of the T i s. Furthermore, in order to maximize the effect of minimization, T should be the minimal hitting set of the T i ...

show abstract

Section: Testing Requirementsmentioning

confidence: 99%

Regression testing minimization, selection and prioritization: a survey

Yoo¹,

Harman²

2012

Softw. Test. Verif. Reliab.

1,058

594

View full text Add to dashboard Cite

show abstract

“…The main limitation of these approaches comes from the state explosion problem and the test case explosion problem [32]. Several proposals address these limitations, for example on the basis of the formalization of reachable properties guiding the test generation or by reducing the test suites using individual requirements selection [33].…”

Section: Related Work and Discussionmentioning

confidence: 99%

Generation of test sequences from formal specifications: GSM 11‐11 standard case study

et al. 2004

View full text Add to dashboard Cite

This paper presents the results of a case study on generating test cases for a fragment of the smart card GSM 11-11 standard. The generation method is based on an original approach using the B notation and techniques of constraint logic programming with sets. The GSM 11-11 technical specifications were formalized with the B notation. From this B specification, a system of constraints was derived, equivalent to this formal model. Using a set constraint solver, boundary states were computed and test cases were obtained by traversing the constrained reachability graph of the specifications. The purpose of this project was to evaluate the contribution of this testing environment, called B-TESTING-TOOLS, in an industrial process on a real life-size application, by comparing the generated test sequences with the already used and high-quality manually-designed tests. This comparison enabled us to validate our approach and showed its effectiveness in the validation process of critical applications: the case study gives a wide coverage (about 85%) of the generated tests compared to the pre-existing tests and a saving of 30% in test design time.the specifications are implemented exactly as expected. In application domains of critical software, i.e. software, avionics, railway signalling, automotive control and management systems, secure smart card applications, etc., more than 50% of the development is devoted to the validation of the software product. Testing is performed by applying test cases to an implementation under test, by making observations during the execution of the tests, and by subsequently assigning a verdict on the correct execution of the implementation. The design of black-box tests in industry is seldom supported by tools and relies on the expertise of validation engineers. Consequently, there is considerable subjectivity exercised in created test sets. The manual design of functional black-box tests is generally based around an informal review of the system specification.The need to offer better methods and tools for functional black-box testing has given rise to a large amount of research on generating tests from formal specifications, e.g. [1][2][3]. Formal methods of specification, particularly model-oriented notations such Z [4], VDM [5] and B [6], allow a high-level abstract formalization of the expected behaviour of the system under test. These notations are well suited for test generation because the expressiveness of set-oriented logic constructs and the definition of an explicit model help both test case generation and oracle synthesis. Thus, these formal notations are the basis of various proposals to more or less automatically generate tests from the formal model, see for example [7][8][9][10][11].In [12][13][14], we presented a new method for automated test generation from B abstract machines using constraint logic programming [15], called B-TESTING-TOOLS. From the B formal model of the functional requirements of the system to be tested, an equivalent system of constraints is derived and, th...

show abstract

“…Code-based techniques for RTS selection (see, for example, 3-5), reduction (see, for example, [6][7][8], and prioritization (see, for example, [9][10][11] are based on the changes in the source code of the SUT. On the other hand, model-based (specification-based or requirements-based) techniques for RTS selection (see, for example, [12][13][14], reduction (see, for example, [15][16][17], and prioritization (see, for example, [18][19][20] are based on the changes on the model of the SUT.…”

Section: Introductionmentioning

confidence: 99%

“…Some model-based techniques for regression testing utilize the concepts of dependence analysis, namely, data dependence and control dependence to define dependencies that capture the effects of the modifications on the model of the SUT [15][16][17][19][20][21][22][23]. Basically, data dependence exists between two statements m and n when a value assigned to a variable v at m reaches n where v is referenced.…”

Section: Introductionmentioning

confidence: 99%

“…The use of dependence analysis to identify dependencies that capture the effects of the modifications on the model of a SUT first appeared in the work on RTS reduction of Korel et al [15,16]. In that work, the effects of each modification (i.e., addition or deletion of a transition in a given EFSM representation of the requirements of a SUT) are captured in terms of affecting, affected, and side effect dependencies between transitions of the EFSM.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Regression test suite selection using dependence analysis

Ural

Yenigün

2012

J Software Evolu Process

View full text Add to dashboard Cite

Dependence analysis on an Extended Finite State Machine representation of the requirements of a system under test identifies various types of control and data dependencies between transitions caused by a set of modifications on the requirements. These particular types of dependencies capture the effects of the modifications, that is, their direct effects on the changed parts of the system and their side effects on the unchanged parts of the system. Recent work on model-based regression testing shows that dependencies capturing direct effects and side effects of the changes made on the requirements can be used for regression test suite (RTS) reduction (reducing the size of a given test suite by eliminating redundancies), for RTS prioritization (ordering test cases in a given test suite for early fault detection), or for RTS generation (designing a test suite covering the identified dependencies). This paper proposes an additional use of such dependencies, namely, RTS selection, which is the process of selecting a subset of a given test suite to form an RTS by considering the coverage of dependencies related to the effects of the modifications. The dependencies marked during this process as uncovered provide a basis for augmenting an (incomplete) RTS with test cases covering uncovered dependencies.

show abstract

Dependence analysis in reduction of requirement based test suites

Cited by 35 publications

References 20 publications

Regression testing minimization, selection and prioritization: a survey

Regression testing minimization, selection and prioritization: a survey

Generation of test sequences from formal specifications: GSM 11‐11 standard case study

Regression test suite selection using dependence analysis

Contact Info

Product

Resources

About