State of the art in impact assessment of policy-oriented international agricultural research

Abstract

Policy-oriented research (POR) represents an increasing share of the Consultative Group on International Agricultural Research’s (CGIAR’s) portfolio. A literature inventory finds 24 studies of the diffusion, influence or impact of this research, of which only three assess impact. There are many influences behind policy formulation, through which research insights permeate, if influential. Hence, attribution and counterfactual difficulties have led many assessments to focus only on analysis of the processes of creating policy influence, rather than also assessing impacts from resulting policy changes. To satisfy CGIAR investor expectations for documented impacts, approaches are proposed for ‘demand-led’ and ‘supply-led’ POR case studies.

Assessing the impact of CIFOR’s influence on policy and practice in the Indonesian pulp and paper sector

“…As Raitzer and Ryan (2008) note, there are only a handful of studies that have successfully identified the economic benefits of policy oriented research, which renders this a rather unique case. Moreover, most of the estimates from the present study lie at the upper end of prior estimates identified in Alston et al 's (2000) comprehensive meta analysis of studies of the economic impact of technology oriented agricultural research.…”

“…Given that there are no objective indices for the influence of information on policy decisions, a qualitative key informant interview approach is necessary to explore the information considered by key individuals involved in the impact pathways (SPIA 2006;Raitzer and Ryan 2008). These interviews were generally conducted in the sequence of the impact pathway, with direct users of the research interviewed first, followed by those who have been influenced by the users.…”

“…Given scarce resources for research, this raises the question of whether such investment merits the opportunity costs of foregone investment in technology oriented activities. A definitive answer has not been possible previously, due to well documented constraints to the attribution of the policy influence of research-derived information (SPIA 2006;Raitzer and Ryan 2008).…”

“…Despite dozens of attempts to discern the impact of policy oriented research in the agricultural and natural resource sectors, only a few studies have successfully estimated quantitative measures of research benefits (SPIA 2006;Raitzer and Ryan 2008).…”

“…In their review of studies of the diffusion, influence and impact of international agricultural policy oriented research, Raitzer and Ryan (2008) find that most prior studies have not been clear about the evidence underlying claims of causal effects of research. The vast majority of studies lack explicit counterfactual scenarios, a cornerstone of good impact assessment practice (Maredia et al 2000), and all but a few have not investigated the benefits or costs of the policy changes identified.…”