Temporal correctness is crucial for real-time systems. Few methods exist to test temporal correctness and most methods used in practice are ad-hoc. A problem with testing real-time applications is the responsetime dependency on the execution order of concurrent tasks. Execution order in turn depends on execution environment properties such as scheduling protocols, use of mutual exclusive resources as well as the point in time when stimuli is injected. Model based mutation testing has previously been proposed to determine the execution orders that need to be verified to increase confidence in timeliness. An effective way to automatically generate such test cases for dynamic real-time systems is still needed. This paper presents a method using heuristic-driven simulation to generate test cases.

Temporal correctness is crucial to the dependability of real-time systems. Few methods exist to test for temporal correctness and most existing methods are ad-hoc. A problem with testing real-time applications is the dependency on the execution time and execution order of individual tasks. Thus, the response times for the tasks may be non-deterministic with respect to inputs. Conventional test coverage criteria ignore task interleaving and timing and, thus do not help determine which execution orders need to be exercised to test for temporal correctness. This paper presents test criteria based on mutation to test timeliness. We also show how previously proposed methods in specification based testing can be applied to testing real-time systems.

No abstract

A problem with testing timeliness of real-time applications is the response-time dependency on the execution order of concurrent tasks. Conventional test methods ignore task interleaving and timing and thus do not help determine which execution orders need to be exercised to test temporal correctness. Model based mutation testing has been proposed to generate inputs and determine the execution orders that need to be verified to increase confidence in timeliness. This paper evaluate a mutation-based framework for automated testing of timeliness by applying it on a small control system running on Linux/RTAI. The experiments presented in this paper indicate that mutation-based test cases are more effective than random and stress tests in finding both naturally occurring and randomly seeded timeliness faults.

Temporal correctness is crucial for the dependability of real-time control systems. A problem with testing such systems is the dependency on the execution orders oftasks. Mutation-based testing criteria have been proposed to determine which execution orders need to be exercised to verify that real-time systems are timely. For flexible control systems, timeliness in itselfmay only be relevantfor a sub-set of tasks, whereas maintained control performance in the presence of worst-case jitter and disturbances is essential. This paper presents an extension to the co-simulator tool TrueTime, to support mutation-based testing of control performance and timeliness. Further, an approach for automatic generation of test cases using genetic algorithms is presented. A conclusion is that testing criteria for timeliness can be used to increase confidence in the dependability offlexible control systems.