Adaptive testing of controllers for autonomous vehicles

Abstract

Autonomous vehicles are likely to require sophisticated software controllers to maintain vehicle performance in the presence of vehicle faults. The test and evaluation of complex software controllers is expected to be a challenging task. The goal of this effort is to apply machine learning techniques from the field of artificial intelligence to the general problem of evaluating an intelligent controller for an autonomous vehicle. The approach involves subjecting a controller to an adaptively chosen set of fault scenarios within a vehicle simulator, and searching for combinations of faults that produce noteworthy performance by the vehicle controller. The search employs a genetic algorithm. We illustrate the approach by evaluating the peiformance of a subsumption-based controller for an autonomous vehicle. The preliminary evidence suggests that this approach is an effective alternative to manual testing of sophisticated software controllers.

Formulation and Theoretical Analysis of the Control Algorithm of the Autonomous Vehicle

“…fuzzy logic), Li, Nutter (1990) dirbtinių neuroninių tinklų (angl. neural nets), Schultz et al (1992) -genetinio programavimo (angl. genetic algorithm) metodais pagrįstus algoritmus.…”

A Genetic Algorithm Approach to Focused Software Usage Testing

“…While, the GA approach presented in this paper is unlike other published approaches to the application of GA to support software testing or software quality assessment, the "failure-pursuit sampling" work of [Dickinson et al 2001] and the "adaptive testing" work of [Schultz et al 1992] are particularly noteworthy with respect to their motivation for the work of this paper.…”

“…[ Schultz et al 1992] demonstrated the use of adaptive testing to test intelligent controllers for autonomous vehicles by creating individuals in the population that represented fault scenarios to be supplied to simulators of the autonomous vehicles. A benefit of such testing was to provide more information to the developers.…”

“…A benefit of such testing was to provide more information to the developers. According to [Schultz et al 1992], "In more of a qualitative affirmation of the method, the original designer of the AUTOACE intelligent controller was shown procedure GA { initialize population; while termination condition not satisfied do { evaluate current population; select parents; apply genetic operators to parents to create children; set current population equal to be the new child population; } } some of the interesting scenarios generated by the GA, and acknowledges that they gave insight into areas of the intelligent controller that could be improved. In particular, the scenarios as a group tend to indicate classes of weaknesses, as opposed to only highlighting single weaknesses.…”

Using Evolutionary Learning of Behavior to Find Weaknesses in Operating Systems

“…Also, knowledge about the program structure is necessary for Wegener's approach to work, whereas our method does not depend on the program structure at all. [7] also uses evolutionary methods to evolve new tests for a controller, but out of a set of known faults of this controller. Obviously, this is mainly a recombination effort, not the construction of a new threat, as in our approach.…”