In statistical testing, a model is developed to characterize the population of uses of the software, and the model is used to generate a statistically correct sample of all uses of the software. A software ‘usage model’ characterizes the population of intended uses of the software in the intended environment. Statistical testing based on a software usage model ensures that the failures that will occur most frequently in operational use will be found early in the testing cycle. The usage model is based on the software specification. The model can be developed in parallel with the software, thus shortening the elapsed time required to develop the deliver software. Usage modeling has been demonstrated to be an activity that improves the specification, gives an analytical description of the specification, quantifies the testing costs and, with statistical testing, provides a basis from which inferences of software reliability may be made. This paper describes the justification for statistical testing of software using a usage model, describes procedures for developing and using a usage model and discusses several usage modeling issues and recent advances in usage model applications.

Because software system testing typically consists of only a very small sample from the set of possible scenarios of system use, it can be difficult or impossible to generalize the test results from a limited amount of testing based on high-level usage models. It can also be very difficult to determine the nature and location of the errors that caused any failures experienced during system testing (and therefore very difficult for the developers to find and fix these errors). To address these issues, this paper presents a Genetic Algorithm (GA) approach to focused software usage testing. Based on the results of macro-level software system testing, a GA is used to select additional test cases to focus on the behavior around the initial test cases to assist in identifying and characterizing the types of test cases that induce system failures (if any) and the types of test cases that do not induce system failures. Whether or not any failures are experienced, this GA approach supports increased test automation and provides increased evidence to support reasoning about the overall quality of the software. When failures are experienced, the approach can improve the efficiency of debugging activities by providing information about similar, but different, test cases that reveal faults in the software and about the input values that triggered the faults to induce failures.

No abstract

No abstract

The ideas and findings in this report should not be construed as an official DoD position. It is published in the interest of scientific and technical information exchange.