Building on experiences collected in the course of an evaluation of a German cluster policy programme, the Leading-Edge Cluster Competition (LECC, evaluation 2008(LECC, evaluation -2014, this paper scrutinizes central problems that arise in the evaluation of the impacts of large-scale innovation policy programmes. We find that our relatively modest knowledge with regard to the actual effects and impact patterns of comparable programmes is not necessarily due to methodological weaknesses of the evaluation studies, but rather to inherent structural programme characteristics. The present state-of-the-art evaluation methodology does not sufficiently allow us to take into consideration these characteristics. Challenges in the evaluation of technology programs relate closely to different facets of complexity. Our analysis shows that three aspects of programme effects deserve more attention in evaluation research: Emergence and non-linearity, uncertainty, and time patterns of the observed effects. Using the example of the LECC, our paper demonstrates how evaluators' work is affected by these phenomena. The results lead to the question of whether there are methodological alternatives that are suitable for future evaluations of complex innovation programmes.

Based on empirical findings on the effects of cluster policies in Germany, this paper scrutinises the available knowledge on cluster policies impact. There is a growing body of insights on direct effects of policy measures on cluster actors, cluster organisations and innovation networks of the promoted clusters. For some industries such as biotechnology, there are indications that cluster policies had a substantial influence on the formation of new firms and emerging sectoral structures. While the available information seems to support the hypothesis that cluster policies can provide positive impulses for the development of clusters, the actual knowledge on far-reaching impacts of cluster policies on economic structures and processes is still rather limited. The paper asks for the reasons of this knowledge gap between expectations placed in cluster policies and the available evidence on their impact. We identify five reasons: (i) problems in addressing the systemic nature of cluster policy interventions and their effects, (ii) deficiencies regarding the methodologies used, (iii) a lacking informational basis, (iv) practical contexts (e.g., a lack of interest of policy makers) leading to deficiencies in incentive mechanisms and (v) the limited transferability of evaluation results to other cluster policy contexts. For future evaluations, we propose among others the use of system-related approaches to impact analyses based on mixed-method designs as well as comparative case studies based on new methods like process tracing. In order to improve the incentives for evaluators, an increasing awareness of policy makers about the relevance of evaluation studies would be important.

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. summary: Innovation policies in both old and new industrial countries increasingly promote regional clusters and distributed networks. Public financing of cluster and networks of business firms, research institutions and other organizations is based on the observation that networking plays an increasing role in technological progress. This paper analyses the German cluster and networks programs which receive funding by the Federal Government or the Federal states. About 300 state-sponsored cluster and network initiatives currently address JEL Classification: O33, O38, O25 → Terms of use: Documents in

In Germany, industrial collective research (ICR) provides a unique framework for research collaborations: an industry-supported network of firms and research institutes conducts research for firms in low- and medium-technology branches. The research projects are mainly financed by a publicly funded program. Based on two surveys, one for research institutes and one addressing firm representatives, we analyze for the first time the institutional features and interactions in ICR. We ask how business firms are involved in network activities and how they benefit from the knowledge created. The results from research in ICR are usually relevant for several firms (e.g. results related to new norms and standards). The network also provides a framework for research on high-tech applications by enabling collaboration across different sectors and technology fields. ICR has proven rather successful in achieving the balancing act between aims of the network and diverging interests of the actors.Innovation networks, industrial collective research, low- and medium-technology sectors,

This article shows that process tracing developed in social science research can be used in evaluations of complex structural and technology policy programmes to overcome deficits in the methodological instruments used to date. Cluster policies are a well-suited example because they are characterized by complex impact patterns like many other current structural and innovation policy programmes. The origin and characteristics of the methodological approach of process tracing are discussed and weaknesses of impact evaluations of cluster programmes highlighted. Subsequently, we look at the potentials of process tracing in impact evaluations of cluster programmes within the framework of mixed-method designs. Our analysis shows that process tracing can enrich the applied methodological repertoire. It allows the evaluators to test the accuracy of the theoretical assumptions underlying the analysed programme and to identify causal mechanisms.