Using Property-Based Oracles when Testing Embedded System Applications

Abstract

Abstract-Embedded systems are becoming increasingly ubiquitous, controlling a wide variety of popular and safetycritical devices. Effective testing techniques could improve the dependability of these systems. In prior work we presented an approach for testing embedded systems, focusing on embedded system applications and the tasks that comprise them. In this work we focus on a second but equally important aspect of testing embedded systems; namely, the need to provide observability of system behavior sufficient to allow engineers to detect failures. We present several property-based oracles that can be instantiated in embedded systems through program analysis and instrumentation, and can detect failures for which simple output-based oracles are inadequate. An empirical study of our approach shows that it can be effective.

Automated Test Oracles

“…Rules can be supported directly either by the execution environment or platform or by the test oracles that generate code to explicitly collect relevant values and events and check them against the rules. For example, property-based oracles are essentially general rules of embedded programs about semaphores, message queues, and critical sections [79]. The technique generates oracles from these rules by instrumenting the program code, the operating system, and the libraries to generate runtime traces that are analyzed offline.…”

An observable and controllable testing framework for modern systems

“…We have developed a family of internal oracles for embedded systems that use instrumentation to record various aspects of execution behavior and compare observed behavior to certain intended system properties that can be derived through program analysis [17]. We focus on several attributes related to proper uses of synchronization primitives and other mechanisms important to managing cooperation and concurrency among tasks (e.g., semaphores, message passing protocols, critical sections, and monitors).…”

Examination of the Possibilities for Integrated Testing of Embedded Systems

“…It is always necessary to assess the correctness of the decision and the basis on which it was taken. Test Oracles are based on the tested object properties [20] or on the output results [21]. Test Oracles relying on the output results for some data combinations have predictable outputs and checks whether the received output value corresponds to the expected one.…”