Evidence Synthesis as the Basis for Decision Analysis: A Method of Selecting the Best Agricultural Practices for Multiple Ecosystem Services

Abstract

Agricultural management practices have impacts not only on crops and livestock, but also on soil, water, wildlife, and ecosystem services. Agricultural research provides evidence about these impacts, but it is unclear how this evidence should be used to make decisions. Two methods are widely used in decision making: evidence synthesis and decision analysis. However, a system of evidence-based decision making that integrates these two methods has not yet been established. Moreover, the standard methods of evidence synthesis have a narrow focus (e.g., the effects of one management practice), but the standard methods of decision analysis have a wide focus (e.g., the comparative effectiveness of multiple management practices). Thus, there is a mismatch between the outputs from evidence synthesis and the inputs that are needed for decision analysis. We show how evidence for a wide range of agricultural practices can be reviewed and summarized simultaneously ("subject-wide evidence synthesis"), and how this evidence can be assessed by experts and used for decision making ("multiple-criteria decision analysis"). We show how these methods could be used by Shackelford et al. Evidence Synthesis for Decision Analysis The Nature Conservancy (TNC) in California to select the best management practices for multiple ecosystem services in Mediterranean-type farmland and rangeland, based on a subject-wide evidence synthesis that was published by Conservation Evidence (www. conservationevidence.com). This method of "evidence-based decision analysis" could be used at different scales, from the local scale (farmers deciding which practices to adopt) to the national or international scale (policy makers deciding which practices to support through agricultural subsidies or other payments for ecosystem services). We discuss the strengths and weaknesses of this method, and we suggest some general principles for improving evidence synthesis as the basis for multi-criteria decision analysis.

Bird‐friendly wine country through diversified vineyards

“…2019; Shackelford et al. 2019). In a global meta‐analysis of working landscapes (including croplands, rangelands, and forests), Prevedello et al.…”

Dynamic meta-analysis: a method of using global evidence for local decision making

“…For example, we showed how a hypothetical user could reach a different conclusion when using the global evidence (cover crops have no effect on cash crop yields) instead of the locally relevant evidence (brassicas have negative effects on cash crop yields in California). As a next step, this evidence could be used as an input into decision analysis [ 13 ], but that is beyond the scope of our work here. Here we discuss some strengths and weaknesses of dynamic meta-analysis, and we suggest that this method could be scaled up and used for subject-wide evidence synthesis in several scientific disciplines.…”

“…Subject-wide evidence synthesis is evidence synthesis on the scale that is needed for multi-criteria decision analysis [ 13 ], and thus it is particularly relevant to a discussion of evidence-based decision making. Because subject-wide evidence synthesis is global in scale, it begs the question, “How relevant is this global evidence for my local decision?” We suggest that dynamic meta-analysis, or some similar method of assessing the local relevance of global evidence, could be especially useful for subject-wide evidence synthesis.…”

“…Decision-support systems enable users to weight evidence by value to stakeholders (e.g. [ 13 ]). However, as far as we are aware, there are no tools that enable users to both filter and weight the studies in a meta-analysis, and thereby to answer the question, “How effective is this intervention in my specific context?” Therefore, we developed a tool for this purpose, and here we show how this tool could be used to assess the local relevance of a global meta-analysis in agroecology.…”

Supporting evidence varies for rangeland management practices that seek to improve soil properties and forage production in California

“…We performed a subject-wide evidence synthesis on data derived exclusively from California rangelands. A subject-wide evidence synthesis is a way to review and summarize the effects of multiple practices or interventions simultaneously using some combination of qualitative and quantitative methods, and it can be a cost-effective approach to support management and policy decision making (Shackelford et al 2019). In our synthesis, we extracted soil and plant data from relevant studies that fulfilled specific requirements and aggregated similar response metrics into umbrella categories (e.g., the cations category combines values of individual exchangeable cations, base saturation and cation exchange capacity).…”