A Study on a Path-Based Strategy for Selecting Black-Box Generated Test Cases

Abstract

Various black-box methods for the generation of test cases have been proposed in the literature. Many of these methods, including the category-partition method and the classification-tree method, follow the approach of partition testing, in which the input domain is partitioned into subdomains according to important aspects of the specification, and test cases are then derived from the subdomains. Though comprehensive in terms of these important aspects, execution of all the test cases so generated may not be feasible under the constraint of tight testing resources. In such circumstances, there is a need to select a smaller subset of test cases from the original test suite for execution. In this paper, we propose the use of white-box information to guide the selection of test cases from the original test suite generated by a black-box testing method. Furthermore, we have developed some techniques and algorithms to facilitate the implementation of our approach, and demonstrated its viability and benefits by means of a case study. . Downloaded from www.worldscientific.com by TEXAS CHRISTIAN UNIVERSITY on 02/05/15. For personal use only. 114 Y. T. Yu et al.due to the huge size of the input domain. Thus, a more practical approach is to construct a test suite, that is, a subset of the input domain for testing [2,3].The construction of a test suite is a critical component of the testing process, since it affects the scope and comprehensiveness of the test. Moreover, it largely determines the amount of testing resources required. Ideally, a test suite has to satisfy many requirements, but in reality some of these requirements may be conflicting. For example, the test suite should be as comprehensive as possible so that it is effective in detecting faults in the software, and it should be as small as possible in order to control the cost of the testing. A very comprehensive test suite containing a large number of test cases can be too costly to be practical, while a small but inadequate test suite may fail to detect some of the faults that exist.Various testing strategies have been proposed for systematic construction of a test suite. These strategies have been broadly classified as white-box (or code-based ) testing or black-box (or specification-based ) testing. The former refers to testing based on the information derived from the source code of the program, whereas the latter makes use of information from the specification. Both white-box and blackbox testing have their own merits and limitations; they are generally considered complementary to each other [4][5][6]. In principle, a testing methodology based on either black-box or white-box information alone is necessarily incomplete.Generally, white-box testing requires the test cases to execute the various components of the program code. Typical examples are (a) branch and path testing [7], which are based on the program's control flow structure, (b) data-flow testing [8,9], which exploits the definition-use associations of the program variables, and (c) mutation tes...

Contributions of tester experience and a checklist guideline to the identification of categories and choices for software testing

“…Software quality is of the utmost importance to both developers and users of software systems. Testing remains the most practical means of assuring the quality of software (Bache and Müllerburg 1990;Yu et al 2001).…”

On the testing methods used by beginning software testers

“…Comments by students involved in the case studies suggest that CTM can be strengthened by the provision of more guidelines on the identification of classifications and classes, integrating CTM with white box testing methods [10], and using the method early in the software life cycle for other defect detection activities such as reviews and inspection [2]. Further work in these directions is definitely worthwhile to provide more practical assistance and guidelines to software practitioners.…”

On the Coverage of Program Code by Specification-Based Tests

“…It is well understood that both black box and white box testing have their own merits and limitations; they are complementary to each other and, hence, both are needed [5,6,22,28]. For example, Richardson and Clarke [22] developed a partition analysis method that combines specification and code information for testing and verification.…”

“…Chen et al [6] proposes an integrated approach for object-oriented program testing. Chan and Yu [5] and Yu et al [28] use a path-based strategy to select test cases that are generated by means of black-box methods.…”

“…In practice, since the software specification is normally available early in the development life cycle, it can be used as the basis to systematically generate a comprehensive initial test suite without requiring the implementation details [5,28]. The test suite is also instrumental in facilitating early discovery of defects [9], saving the high costs incurred due to latent defects that persist through to the operational software.…”