Evolutionary Safety Testing of Embedded Control Software by Automatically Generating Compact Test Data Sequences

Abstract

Whereas the verification of non-safety-related, embedded software typically focuses on demonstrating that the implementation fulfills its functional requirements, this is not sufficient for safety-relevant systems. In this case, the control software must also meet application-specific safety requirements.Safety requirements typically arise from the application of hazard and/or safety analysis techniques, e.g. FMEA, FTA or SHARD. During the downstream development process it must be shown that these requirements cannot be violated. This can be achieved utilizing different techniques. One way of providing evidence that violations of the safety properties identified cannot occur is to thoroughly test each of the safety requirements. This paper introduces Evolutionary Safety Testing (EST), a fully automated procedure for the safety testing of embedded control software. EST employs extended evolutionary algorithms in an optimization process which aggressively tries to find test data sequences that cause the test object to violate a given safety requirement.A compact description formalism for input sequences for safety testing is presented, which is compatible with description techniques used during other test process stages. This compact description allows 1) an efficient application of evolutionary algorithms (and other optimization techniques) and 2) the description of long test sequences necessary for the adequate stimulation of real-world systems. The objective function is designed in such a way that optimal values represent test data sequences which violate a given safety requirement. By means of repeated input sequence generation, software execution and the subsequent evaluation of the objective function each safety requirement is extensively tested.The use of EST for the safety testing of automotive control software is demonstrated using safety requirements of an adaptive cruise control (ACC) system.The EST approach can easily be integrated into an overall software test strategy which combines different test design techniques with specific test objectives.

A systematic review of search-based testing for non-functional system properties

“…Most of our developed criteria were fulfilled by all of the studies, exceptions being [34,35,12,15,54] where evidence of a comparison group was missing, but these quality differences were not found to be largely confounded with study outcomes. What follows next is the list of developed criteria:…”

“…In Baresel et al [15,54], an evolutionary testing approach using genetic algorithms was presented for structural and functional sequence testing. For complex dynamic system like car control systems, long input sequences were necessary to simulate these systems.…”

“…More experimentation is required to confirm the efficiency applying revised SA algorithm. Baresel et al [15], and Pohlheim et al [54] GA Problem specific fitness function measuring different properties e.g. signal amplitude boundaries…”

“…In Wegener et al [25], genetic algorithms (GA) were used to search for input situations that produce very long or very short 2005 [54] execution times. The fitness function used was the execution time of an individual measured in micro seconds.…”

Untersuchungen zur Parallelverarbeitung mit Wissenschaftlich-technischen Berechnungsumgebungen

“…Der Unterschied der Struktur gegenüber Abbildung 7.5 ist, dass sowohl Gütefunktion als auch Simulationsanwendung hier lediglich auf lokale Variablen zugreifen. Eine detaillierte Beschreibung des Anwendungsproblems sowie der sequentiellen Implementierung findet sich in[88]. parallel abzuarbeitenden Teilaufgaben entsprechen in diesem Anwendungsfall den Gütewertberechnungen unterschiedlicher Individuen innerhalb eines Iterationsschrittes.Während der Abarbeitung einer Teilaufgabe ist keine Koordination mit anderen Instanzen notwendig, während die Teilaufgaben mehrfach, das heisst zu jedem Iterationsschritt des Optimierungsalgorithmus, abzuarbeiten sind.…”