Science informing Policy – a health warning for the environment

Pullin, Andrew S.; Knight, Thomas Andrew

doi:10.1186/2047-2382-1-15

Cited by 20 publications

(17 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Globally, our results indicate limited transparency and reproducibility of some meta-analyses. Low-quality meta-analyses are strongly criticized (Kirsch et al 2008, Pullin and Knight 2012, Ioannidis 2016) and negatively impact the image of usefulness of such method (Whittaker 2010). Our in-depth quality assessment could serve as a benchmark to perform new meta-analyses.…”

Section: Quality Of the Meta-analysesmentioning

confidence: 99%

Evidence map of crop diversification strategies at the global scale

Beillouin

Ben-Ari

Makowski

2019

Environ. Res. Lett.

106

View full text Add to dashboard Cite

The diversification of cropping systems encompasses different strategies that may help maintain or enhance the sustainability of agriculture. Thousands of experiments have been carried out around the world since almost five decades to evaluate and compare the performances of various diversification strategies in a wide array of agroecosystems and climates. Although these analyses have been synthesized in a growing number of meta-analyses, the information remains somewhat fragmented. A multicriteria systematic synthesis of worldwide agricultural diversification is still lacking. Here, we review all meta-analyses conducted on crop diversification strategies and produce a detailed overview of their results and of their quality. We identified and analyzed 99 meta-analyses summarizing the results of more than 3700 agronomic experiments on seven crop diversification strategies. Among these strategies, rotation and associated plant species are dominant in the literature followed by intercropping, agroforestry and landscape heterogeneity. Our analysis reveals that rotation and intercropping are associated with yield increases. Agroforestry systematically induces an improvement of biodiversity and soil quality-in particular soil organic carbon. We show that, regardless of the context, a combination of several diversification strategies outperforms any individual strategy. Our review reveals that a significant knowledge gap remains, in particular regarding water use, farmers' profitability, product quality and production stability. Few meta-analyses investigate the performance of landscape heterogeneity and of systems with species other than cereals and pulses. Additionally, we show that most of the meta-analyses studied cannot be considered fully transparent and reproducible. Their conclusions should therefore be interpreted with caution. Our systematic mapping provides a benchmark to guide and improve the relevance and reliability of future meta-analyses in agronomy.

show abstract

Section: Quality Of the Meta-analysesmentioning

confidence: 99%

Evidence map of crop diversification strategies at the global scale

Beillouin

Ben-Ari

Makowski

2019

Environ. Res. Lett.

106

View full text Add to dashboard Cite

show abstract

“…These qualities also assist researchers in identifying gaps in knowledge and areas of controversy or uncertainty, and can help decision-makers undertake informed management and defend potentially controversial or expensive actions (Gough et al, 2012). Where these qualities are not present, reviews have the potential to misinform and result in policies that have unwanted and unforeseen consequences and/or wasted research investment (Kirsch et al, 2008;Pullin and Knight, 2012), particularly if used as the single source of knowledge (although this will rarely be the case) or if selectively used by stakeholders with particular priorities. Avoiding such an eventuality imposes an obligation on those conducting evidence reviews to ensure their reliability and accurate reflection of the primary evidence base, and to transparently report review methodology to enable external assessment of reliability.…”

Section: Introductionmentioning

confidence: 99%

“…Claudet et al, 2008;Stewart et al, 2009) and misrepresentation of data within reviews resulting from conflicts of interest with funding organisations has been indicated (Wade et al, 2010). Perhaps more commonly, selection of primary data to support an adopted position or belief (so--or subconsciously employed by review authors (Biber, 2012;Pullin and Knight, 2012). While the translation of evidence from science to policy is rarely linear and decisions are informed through other mechanisms as well as published literature (e.g.…”

Section: Introductionmentioning

confidence: 99%

The reliability of evidence review methodology in environmental science and conservation

O‘Leary

Kvist

Bayliss

et al. 2016

Environmental Science & Policy

View full text Add to dashboard Cite

Given the proliferation of primary research articles, the importance of reliable environmental evidence reviews for informing policy and management decisions is increasing. Although conducting reviews is an efficient method of synthesising the fragmented primary evidence base, reviews that are of poor methodological reliability have the potential to misinform by not accurately reflecting the available evidence base. To assess the current value of evidence reviews for decision-making we appraised a systematic sample of articles published in early 2015 (N=92) using the Collaboration for Environmental Evidence Synthesis Assessment Tool (CEESAT). CEESAT assesses the methodology of policy-relevant evidence reviews according to elements important for objectivity, transparency and comprehensiveness. Overall, reviews performed poorly with a median score of 2.5/39 and a modal score of zero (range 0-30, mean 5.8), and low scores were ubiquitous across subject areas. In general, reviews that applied meta-analytical techniques achieved higher scores than narrative syntheses (median 18.3 and 2.0 respectively), as a result of the latter consistently failing to adequately report methodology or how conclusions were drawn. However, some narrative syntheses achieved high scores, illustrating that the reliability of reviews should be assessed on a case-by-case basis. Given the potential importance of reviews for informing management and policy, as well as research, it is vital that overall methodological reliability is improved. Although the increasing number of systematic reviews and meta-analyses highlight that some progress is being made, our findings suggest little or no improvement in the last decade. To motivate progress, we recommend that an annual assessment of the methodological reliability of evidence reviews be conducted. To better serve the environmental policy and management communities we identify a requirement for independent critical appraisal of review methodology thus enabling decisionmakers to select reviews that are most likely to accurately reflect the evidence base.

show abstract

“…Within these, they identified multiple or overlapping reviews addressing the same basic issue or question: for example, Claudet et al (2008) and Stewart et al (2009) both reviewed evidence relating to the size of marine reserves. More worrying, Wade et al (2010) explored misrepresentation of data resulting from conflicts of interest with funding organisations, while Biber 2012and Pullin and Knight (2012) both identified a problem with selection of primary data to support an adopted position or belief (so-called policybased evidence).…”

Section: The Role Of Evidencementioning

confidence: 99%

Evidence synthesis in landscape aesthetics: an honourable endeavour yet insufficient applicable knowledge

Bishop

2019

Socio Ecol Pract Res

View full text Add to dashboard Cite

Considerable research effort, over more than 30 years, has been directed toward better understanding of the way people respond to diverse and changing landscapes. Knowledge of human responses to a changing world is an essential ingredient in successful implementation of measures which protect our socio-ecological wellbeing. However, decision-making seldom flows from individual pieces of research. Evidence needs to be synthesised through systematic procedures, including metaanalysis, before having even the potential to influence policy makers. This paper reviews evidence synthesis in the field of landscape aesthetics, especially in the context of wind energy and forest management. It looks then at the impact of this research, as expressed in guidelines and policies and finds that while there are some meaningful links, the individuality of each landscape change makes empirical findings hard to generalise, beyond a rather superficial level, and their application, especially in a political context, problematic.

show abstract

Science informing Policy – a health warning for the environment

Cited by 20 publications

References 11 publications

Evidence map of crop diversification strategies at the global scale

Evidence map of crop diversification strategies at the global scale

The reliability of evidence review methodology in environmental science and conservation

Evidence synthesis in landscape aesthetics: an honourable endeavour yet insufficient applicable knowledge

Contact Info

Product

Resources

About