Improving policy understanding by means of secondary analyses of policy evaluation

Abstract

This paper is based on the idea that existing evaluations of individual RTDI policy interventions can be utilized to understand policies beyond the respective cases covered by the evaluations. We discuss a methodological approach for a comprehensive secondary analysis of evaluations. With this approach, existing evaluations can be used to learn not only about policy interventions, but also about policy performance and policy effects on the system level

Políticas de innovación inclusiva y tensiones en torno a su implementación y evaluación en Colombia: el caso de A Ciencia Cierta

“…Las aproximaciones para evaluar las políticas de innovación son, en efecto, diversas y dependen de los enfoques de política dominantes. Si bien es posible distinguir varias corrientes evaluativas de la CTeI 5 , aquí nos centramos en la aproximación de evaluación formativa (Magro y Wilson, 2013), de creciente interés en la literatura para valorar el mérito de nuevos marcos de política de inno-5 Recomendamos los trabajos de Georghiou (1999), Arnold (2004), y Edler et al (2008;2016), entre otros.…”

Science-industry knowledge exchange

“…So far, typical evaluations focus on individual policy instruments in isolation, without considering how different instruments interact within a policy mix (Borrás and Laatsit, 2018;Edler et al, 2012). Greater efforts are necessary to move towards evaluation methods that consider the combined effects of policy instruments, as well as potential redundancies, contradictions and remaining problems that could be addressed with new instruments (Edler et al, 2008;Magro and Wilson, 2013).This could be done by more systematic evaluations of entire policy mixes and by introducing, within the templates used to evaluate individual policy instruments, a specific section that focusses on their interaction with the broader policy mix.…”

The practice of evaluation in innovation policy in Europe

“…The objectives and intervention mechanisms of innovation policy have broadened in scope. Indeed, innovation policy is in fact a mix of policies and is itself a more or less integral part of a broader policy portfolio at various levels (Flanagan and Uyarra 2011, Edler 2008, Arnold 2004. Intervention rationales in innovation policy are based on a set of theoretical assumptions as to what drives innovation capabilities and performance, and how improved capabilities and performance lead to technological, environmental, social and economic impacts, which illustrates the high expectations regarding the effectiveness of innovation support measures.…”

“…It uses techniques of Meta-Evaluation 2 to assess the overall design, implementation and functionality of evaluations to learn about evaluation itself, not about the impacts of the underlying policies (Implore 2009). It does not systematically gather and synthesise information from evaluations to better understand policy measures (as described in Edler et al 2008), but analyses evaluations themselves.…”