Repeatability of the searching process in reviews of restoration outcomes

Abstract

In the recently declared United Nations Decade on Ecosystem Restoration (2021–2030), evidence‐based research has an essential role in summarizing restoration outcomes toward broad generalizations to advance restoration practice and science globally. However, to present trustworthy, high‐quality recommendations, evidence reviews should be based on rigorous methods that minimize bias and enhance systematicity, transparency, objectivity, comprehensiveness, and repeatability. To assess the current value of restoration reviews in terms of methodological repeatability at the searching stage, a fundamental review stage, we evaluated a sample of meta‐analyses and narrative syntheses (n = 79) and critically appraised how searches were conducted based on the information reported within each study. By assessing whether review methods were reported in sufficient detail to be repeated by an independent party, we found that repeatability varied substantially by the standard we used. Overall, reviews performed relatively poorly, with a median score of 3 out of 9 points available, and a mean score of 3.6. Most reviews (n = 76/79) failed in effectively reporting all necessary information to allow repeatability at this stage. We found no statistically significant differences considering review types, suggesting that the type of synthesis alone does not solely reflect the repeatability of a review. Toward increasingly repeatable, reliable reviews in restoration ecology, we recommend that authors incorporate and apply principles of systematic reviews and maps in their review process, and suggest that environmental journals should broadly emphasize ways of performing rigorous reviews, as well as increase efforts for the publication of open access review protocols.

Disentangling the importance of intrinsic and extrinsic seed dispersal factors in forest restoration success: a global review

“…We searched for published studies of forest restoration that focused on seed dispersal functions as a key ecological process towards ecosystem restoration. For this review we followed Pickering and Byrne's (2014) recommendations for systematic quantitative approaches to literature reviews, and Romanelli et al (2021) for repeatability of reviews of restoration outcomes. Two global databases were selected to carry out our literature search: Web of Science and Science Direct.…”

Contribution of nucleation techniques to plant establishment in restoration projects: an integrative review and meta‐analysis

“…However, a gradual return of species can minimally guarantee ecosystem services provision, as relatively few species can sustain environmental health due to the high degree of functional redundancy and complementarity between species (Schoolmaster et al 2020). The standardization of metrics is also strongly recommended to increase both the repeatability and the reliability of restoration practices (Pander & Geist 2013; Romanelli et al 2021).…”

The effect of insect food availability on songbird reproductive success and chick body condition: Evidence from a systematic review and meta‐analysis

“…The conclusions drawn from a meta‐analysis are highly dependent on the set of studies included. Following systematic review principles (Pullin et al, 2018), such as pre‐registering a protocol and thoroughly searching the literature, can help ensure reliable findings, yet these steps are often neglected in ecological meta‐analyses (Grames & Elphick, 2020; Romanelli et al, 2021). When designing our literature search (see pre‐registered protocol, Grames, Montgomery, et al, 2019 for full details), we identified three elements that needed to appear in papers as a minimum for them to be considered for our review: (1) a measure of either body condition or reproductive success, (2) a measure of invertebrate food availability or supplementation and (3) data from songbirds (Order: Passeriformes).…”