Automated Test Case Selection for Flight Systems using Genetic Algorithms

Abstract

Without rigorous system verification and validation (SVV), flight systems have no assurances that they will actually accomplish their objectives (e.g., the right system was built) or that the system was built to specification (e.g., the system was built correctly). As system complexity grows, exhaustive SVV becomes time and cost prohibitive as the number of interactions explodes in an exponential or even combinatorial fashion. Consequently, JPL and others have resorted to selecting test cases by hand based on engineering judgment or stochastic methods such as Monte Carlo methods. These two approaches are at opposite ends of the search spectrum, in which one is narrow and focused and the other is broad and shallow. This paper describes a novel approach to test case selection through the use of genetic algorithms (GAs), a type of heuristic search technique based on Darwinian evolution that effectively bridges the search for test cases between broad and narrow spectrums. More specifically, this paper describes the Nemesis framework for automated test case generation, execution, and analysis using GAs. Results are presented for the Dawn Mission flight testbed.

Multi-Objective Test Case Selection: A study of the influence of the Catfish effect on PSO based strategies

“…A promising approach for the TC selection problem is to treat it as a search based optimization problem [Harman 2011]. This waym the search techniques explore the space of possible solutions searching for a TC subset in order to optimize a given objective function, i.e., the given selection criterion (see [Ma et al 2005], [Yoo and Harman 2007], [Barltrop et al 2010], [de Souza et al 2010], [de Souza et al 2013]). In most of the testing environments, there is not only a single selection criterion to be considered.…”