Process Outcome Integration with Theory (POInT): academic report

Abstract

This paper describes the development and testing of a novel approach to evaluating development interventions – the POInT approach. The authors used Bayesian causal modelling to integrate process and outcome data to generate insights about all aspects of the theory of change, including outcomes, mechanisms, mediators and moderators. They partnered with two teams who had evaluated or were evaluating complex development interventions: The UPAVAN team had evaluated a nutrition-sensitive agriculture intervention in Odisha, India, and the DIG team was in the process of evaluating a disability-inclusive poverty graduation intervention in Uganda. The partner teams’ theory of change were adapted into a formal causal model, depicted as a directed acyclic graph (DAG). The DAG was specified in the statistical software R, using the CausalQueries package, having extended the package to handle large models. Using a novel prior elicitation strategy to elicit beliefs over many more parameters than has previously been possible, the partner teams’ beliefs about the nature and strength of causal links in the causal model (priors) were elicited and combined into a single set of shared prior beliefs. The model was updated on data alone as well as on data plus priors to generate posterior models under different assumptions. Finally, the prior and posterior models were queried to learn about estimates of interest, and the relative role of prior beliefs and data in the combined analysis.