Using a constraint satisfaction formulation and solution techniques for random test program generation

Abstract

Automatic generation of test programs plays a major role in the verification of modern processors and hardware systems. In this paper, we formulate the generation of test programs as a constraint satisfaction problem and develop techniques for dealing with the challenges we face, most notably: huge variable domains (e.g., magnitude of 2 64 ) and the need to randomly generate "well distributed" samplings of the solution space. We describe several applications of our method, which include specific test generators targeted at various parts of a design or stages of the verification process.Functional verification is widely acknowledged as the bottleneck of the hardware design cycle. 1 Because simulation is the main vehicle for the functional verification of large and complex designs, stimuli generation for simulation plays a central role in this field. The IBM Haifa Research Laboratory (HRL) has been developing methodology and tools for stimuli generation for more than a decade. These tools aim at designs that are driven by complex micro-architectures, that support rich interface semantics, and that range from processor subunits to large multiprocessor systems.The generated stimuli, usually in the form of test programs, are designed to trigger architecture and micro-architecture events defined by a verification plan. 2 The input for a test program generator is a specification of a test template. An example of such a test template would be a set of tests that exercise the data cache of the processor and that are formed by a series of double-word store and load instructions. The generator produces a large number of distinct well-distributed test program instances that comply with the user's specification. The variation among different instances is achieved through a large number of random decisions made during the generation process. In addition, generated test programs must meet two inherent requirements: (1) tests must be valid, that is, their behavior should be well defined by the specification of the verified system; (2) test programs should also be of high quality, in the sense that they should expand the coverage of the verified system and focus on potential bugs.Early tools developed at HRL demonstrated a biased pseudorandom dynamic generation scheme, coupled with a traditional expert system paradigm. 3 Evolution of the tools led to the model-based test generation scheme, 4 where a generator is partitioned into a generic, system-independent engine and a model that describes the verified system. In recent years, technology has shifted toward constraint-based formulations of the generation task and generation schemes driven by solving constraint satisfaction problems (CSPs). 5 A constraint satisfaction problem consists of a finite set of variables and a set of constraints. Each variable is associated with a set of possible values, known

Constraint-Based Local Search for the Automatic Generation of Architectural Tests

“…For instance, the add and the sub instructions do not use the address and constant variables. As a result, the ATGP gives rise to a Conditional Constraint Satisfaction Problem (CCSP) in the sense of [1], as observed in [5]. Mittal and Falkenhainer showed how to transform a CCSP into a CSP by introducing additional variables denoting whether a variable is active, i.e., whether its condition holds.…”

VLIW - a case study of parallelism verification

“…It was implemented in Genesys-Pro, a high-end IBM test generator [9], by formulating the full VLIW as a single Constraint Satisfaction Problem (CSP) [10].…”

Using Linear Programming Techniques for Scheduling-Based Random Test-Case Generation

“…The requirements from a constraint solver for random stimuli generation are somewhat different [3]. Stimuli generators are required to generate many different solutions from the same test specification.…”

“…Therefore, test case generation plays a central role in this field. In recent years, technology has shifted towards constraintbased modeling of the generation task and generation schemes driven by solving Constraint Satisfaction Problems (CSP) [3]. In fact, leading verification environments, such as Specman [4] and Vera [5], and stimuli generation tools use CSP solvers as the base for the generation engine.…”

“…While this modeling scheme borrows ideas from traditional linear programming modeling techniques for scheduling problems, it provides a new modeling framework. This new modeling framework is designed to address the specific characteristics that are unique to stimuli generation [3]. These characteristics include soft constraints that improve the quality of the generated stimuli and the need to provide random solutions.…”