This article proposes ways to use programme theory for evaluating aspects of programmes that are complicated or complex. It argues that there are useful distinctions to be drawn between aspects that are complicated and those that are complex, and provides examples of programme theory evaluations that have usefully represented and address both of these. While complexity has been defined in varied ways in previous discussions of evaluation theory and practice, this article draws on Glouberman and Zimmerman's conceptualization of the differences between what is complicated (multiple components) and what is complex (emergent). Complicated programme theory may be used to represent interventions with multiple components, multiple agencies, multiple simultaneous causal strands and/or multiple alternative causal strands. Complex programme theory may be used to represent recursive causality (with reinforcing loops), disproportionate relationships (where at critical levels, a small change can make a big difference — a `tipping point') and emergent outcomes.

For over thirty years now, many evaluators have recommended making explicit the underlying assumptions about how programs are expected to work-the program theory-and then using this theory to guide the evaluation. In this chapter, we provide an overview of program theory evaluation (PTE), based on our search to find what was value-added to evaluations that used this approach. We found fewer clear-cut, full-blown examples in practice than expected, but we found many interesting variations of PTE in practice and much to recommend it. And elements of PTE, whether the evaluators use the terminology or not, are being used in a wide range of areas of concern to evaluators. Based on this review, in this chapter we discuss the practice, promise, and problems of PTE. What Is Program Theory Evaluation?Because this volume is intended to demonstrate the diversity of practice, we have used a broad definition of program theory evaluation. We consider it to have two essential components, one conceptual and one empirical. PTE consists of an explicit theory or model of how the program causes the intended or observed outcomes and an evaluation that is at least partly guided by this model. This definition, though deliberately broad, does excIude some versions of evaluation that have the word theory attached to them. It does not cover all six types of theory-driven evaluation defined by Chen (1990) but only the type he refers to as intervening mechanism evaluation. It does not include evaluations that explicate the theory behind a NEW D~KECTIUNS toit EVALUATION, no. 87, Fall 2000 Q Jossey-Bass 5

This chapter highlights the continued salience of Weiss' s questions about theory-based evaluation, especially given the often simplistic uses of program theory in evaluation.

This chapter presents the methodology and findings of a review study on this topic using a realist synthesis approach, which is particularly suited to address complexity issues. The chapter begins with an overview of the review in terms of its scope, processes and findings. It then analyses the review in terms of Pawson's VICTORE framework to demonstrate the complexity of the review topic, how complexity was reflected in the findings, and how the methodology of realist synthesis helped us to manage and deal with complexity. We then discuss how the findings from a realist synthesis can assist in dealing with the complexities of policy and program management in the real world.

No abstract