Evaluation and Validation

Abstract

During the design procedure, we have to check repeatedly whether or not the system under design is likely to perform its function and to satisfy all relevant design objectives. This is the purpose of validations and evaluations which must be performed during the design process. This chapter starts with a presentation of techniques for the evaluation of (partial) designs with respect to objectives. In particular, we consider (worst case) execution time, quality of results, thermal behavior, and dependability as objectives. We provide an introduction into fundamental techniques for computing the worst case execution time. Examples of energy models will be presented in order to demonstrate the need for an adjustment of the level of model details to the particular application at hand. Thermal modeling is reduced to the problem of equivalent electrical modeling. With respect to dependability, an introduction to statistical models of reliability as well as an introduction to fault trees are included. As a means for relating results for the different objectives against each other, we introduce the concept of Pareto optimality. This chapter closes with hints regarding validation techniques, including simulation, rapid prototyping, and formal verification.

Execution trace analysis for a precise understanding of latency violations

“…These requirements can be verified using different approaches throughout a Product Life Cycle (PLC) from design time to runtime. Formal verification approaches use mathematical logic to prove properties [44] [48] [45] while runtime verification approaches detect property violations by monitoring the system during execution [48]. Runtime verification can be applied on traces collected as the system runs (on-line) or afterwards.…”

“…Simulation is a very common technique for evaluating and validating designs as simulation traces record the behavior of an application allocated to an architecture and thus provide relevant information about the system execution. Obviously, this requires models, which are approximations of real systems, to be executable [44].…”