Using process tracing to evaluate unique events: The case of EXPO Milano 2015

Abstract

In the social sciences, there is an emerging interest in process tracing as a method for improving rigour and transparency in within-case inferences. Recently, the method has been proposed as a possible enhancement of theory-based approaches to evaluation, but applications of the method remain rare. In an attempt to fill this gap, process tracing was used to evaluate the Universal Exposition held in Milano in 2015 (EXPO2015). Mega-events of this kind are perfect candidates for ‘testing’ the method; although their effects have been widely discussed in the relevant literature, claims about the causal contribution of mega-events are not straightforward, and a number of ambiguities complicate any clear assessment of their consequences. Two in-depth case studies of projects related to EXPO2015 – the East External Highway and Refettorio Ambrosiano – demonstrate the advantages and feasibility of process tracing and of the application of Bayesian logic to evidence search, collection and assessment. In particular, case study results show that Bayesian scrutiny may reveal unexpected weakness in apparently obvious inferences and increase reliability in assessing less straightforward causal attributions.

Using process tracing to evaluate stakeholder disputes: The case of tender award dispute in Nepal’s forestry

“…Uniqueness illustrates what evidence must be found that cannot be explained with an alternative hypothesis. High uniqueness indicates that the evidence is specific to the hypothesis and unexplained by alternative hypotheses, and low uniqueness means that there are other explanations for finding the piece of evidence (Busetti and Dente, 2017; Mahoney, 2012). Literature has focused on four tests based on a belief in hypothesis or a Bayesian logic, namely the straw in the wind, hoop, smoking gun, and doubly decisive tests (Beach and Pedersen, 2013; Bennett and Checkel, 2012; Collier, 2011).…”

“…It offers a Bayesian approach to evidence search, collection, and assessment. However, the application of this method is still limited (Busetti and Dente, 2017). This study has identified a tender dispute event in a forestry program in Nepal as a fitting candidate for the application of the PT method and to test its usefulness in collecting and assessing evidence for causal claims.…”

Preventing children’s death by heatstroke: How alternative accident theories affect policy design and evaluation

“…To this end, the two policy design processes have been studied applying a congruence analysis. While the methodological debate on research designs based on case studies has recently paid more attention to its applications for ex post evaluation, to test whether or not an hypothesized mechanism is causally connected to a certain outcome (Chen and Henry, 2020; Busetti and Dente, 2017), it can also be applied to ex ante evaluations aimed at assessing the coherence between policy design and goals (Melloni et al, 2016) and the conditions for evaluability (Wauters and Beach, 2018: 289). Different from process tracing, congruence analysis aims at assessing which among several theories better supports certain evidence (Blatter and Haverland, 2012; Wauters and Beach, 2018: 297) and thus is particularly useful to evaluate policy maker’s search for contextually valid solutions when they can choose among theories which are equally valid ‘on average’ (Bennett, 2015), with limited or no opportunities to learn from experience.…”

The Case for Case Studies

“…Thus, a second response in the evaluation community has been increased interest in "process tracing," a method of causal inference that is applicable to single observational case studies (Bamanyaki and Holvoet 2016;Barnett and Munslow 2014;Befani and Mayne 2014;Befani and Stedman-Bryce 2017;Busetti and Dente 2017;Mendoza and Woolcock 2014;Punton and Welle 2015;Schmitt and Beach 2015;Stern et al 2012;Wauters and Beach 2018). Process tracing has been common in political science for decades and has been the subject of recent methodological innovations, most notably the explicit use of Bayesian logic in making inferences about the alternative explanations for the outcomes of cases.…”