Systematic Review and Weight of Evidence Are Integral to Ecological and Human Health Assessments: They Need an Integrated Framework

Abstract

Scientific assessments synthesize the various results of scientific research for policy and decision making. Synthesizing evidence in environmental assessments can involve either or both of 2 systems: systematic review (SR) and weight of evidence (WoE). Systematic review was developed to systematically assemble results of clinical trials to be combined by meta‐analysis. Weight‐of‐evidence approaches have evolved from jurisprudence to make inferences from diverse bodies of evidence in various fields. Our objectives are to describe the similarities and differences between SR and WoE and to suggest how their best practices can be combined into a general framework that is applicable to human health and ecological assessments. Integrating SR and WoE is based on the recognition that 2 processes are required, assembling evidence and making an inference. Systematic review is characterized by methodical literature searching, screening, and data extraction, originally for meta‐analysis but now for various inferential methods. Weight of evidence is characterized by systematically relating heterogeneous evidence to considerations appropriate to the inference and making the inference by weighing the evidence. Systematic review enables the unbiased assembly of evidence from literature, but methods for assembling other information must be considered as well. If only 1 type of quantitative study estimates the assessment endpoint, meta‐analysis is appropriate for inference. Otherwise, the heterogeneous evidence must be weighed. A framework is presented that integrates best practices into a methodical assembly and weighing of evidence. A glossary of terms for the combined practice and a history of the origins of SR and WoE are provided in Supplemental Data. Integr Environ Assess Manag 2020;16:718–728. Published 2020. This article is a US Government work and is in the public domain in the USA.

Toolkit for detecting misused epidemiological methods

“…The exclusive reliance on quantitative methods when qualitative research can provide both a context for the variables included in the quantitative analysis as well as a context for the interpretation of the quantitative findings [85] A17 Producing erroneous or biased meta-analyses and reporting them as representing a weight-of-evidence summary result Meta-analysis includes studies with different study designs, or it selectively excludes studies that should have been included [79] A18 Using mortality instead of morbidity data for a cancer endpoint with a high survival rate For example, using mortality instead of morbidity for breast cancer risk associated with ethylene oxide reduces the risk estimates…”

The Future of the Weight‐of‐Evidence Approach: A Response to Suter's Comments

“…All assessments, like those purporting to be via WoE, should not be accepted at face value without independent scrutiny. To address these and other concerns, we agree with Suter et al (2020) that systematic review is a potentially valuable tool for checking the reliability of studies, but we caution that in medical sciences, where this procedure is used extensively, the reviews themselves are often poorly designed and lacking in quality (Ioannidis, 2016). When deficient, they provide an unsuitable basis for decision-making.…”

Environmental Assessors Should Use State‐of‐Practice Weight‐of‐Evidence Processes

“…As an example of its breadth, I am collaborating with human health risk assessors to apply the WoE guidelines to derivation of human health benchmark values by read-across. Further, because of the growing importance of systematic review for health and environmental assessments, we have integrated it with WoE and clarified the relationship between review and inference (Suter et al, 2020).…”