Dynamic meta-analysis: a method of using global evidence for local decision making

Abstract

Background Meta-analysis is often used to make generalisations across all available evidence at the global scale. But how can these global generalisations be used for evidence-based decision making at the local scale, if the global evidence is not perceived to be relevant to local decisions? We show how an interactive method of meta-analysis—dynamic meta-analysis—can be used to assess the local relevance of global evidence. Results We developed Metadataset (www.metadataset.com) as a proof-of-concept for dynamic meta-analysis. Using Metadataset, we show how evidence can be filtered and weighted, and results can be recalculated, using dynamic methods of subgroup analysis, meta-regression, and recalibration. With an example from agroecology, we show how dynamic meta-analysis could lead to different conclusions for different subsets of the global evidence. Dynamic meta-analysis could also lead to a rebalancing of power and responsibility in evidence synthesis, since evidence users would be able to make decisions that are typically made by systematic reviewers—decisions about which studies to include (e.g. critical appraisal) and how to handle missing or poorly reported data (e.g. sensitivity analysis). Conclusions In this study, we show how dynamic meta-analysis can meet an important challenge in evidence-based decision making—the challenge of using global evidence for local decisions. We suggest that dynamic meta-analysis can be used for subject-wide evidence synthesis in several scientific disciplines, including agroecology and conservation biology. Future studies should develop standardised classification systems for the metadata that are used to filter and weight the evidence. Future studies should also develop standardised software packages, so that researchers can efficiently publish dynamic versions of their meta-analyses and keep them up-to-date as living systematic reviews. Metadataset is a proof-of-concept for this type of software, and it is open source. Future studies should improve the user experience, scale the software architecture, agree on standards for data and metadata storage and processing, and develop protocols for responsible evidence use.

Dominance and prestige: Meta-analytic review of experimentally induced body position effects on behavioral, self-report, and physiological dependent variables.

“…In addition, we explored whether study, participant, and design characteristics moderate the effects. We also provide an app (https://metaanalyses.shinyapps.io/ bodypositions/) for a dynamic meta-analytic review (Shackelford et al, 2021), which allows researchers to continuously update the evidence on this topic.…”

A systematic map of cassava farming practices and their agricultural and environmental impacts using new ontologies: Agri‐ontologies 1.0

“…meta‐analysis; Todman et al, 2023). Hierarchical classification systems such as this enable the automatic lumping and splitting of evidence, using methods such as ‘dynamic meta‐analysis’ (Martin et al, 2022; Shackelford et al, 2021). Agri‐ontologies 1.0 complements existing ontologies, such as DarwinCore (Wieczorek et al, 2012) and AgrO (Aubert et al, 2017).…”

“…if the ‘n.e.c.’ category has many publications coded in it. In particular, these ontologies were designed to develop case studies for ‘dynamic meta‐analysis’, which is a method of meta‐analysis that enables synthesis at multiple levels via an online interactive interface (Shackelford et al, 2020), with the ultimate aim of increasing the relevance and recency of evidence (e.g. via ‘living systematic reviews’) and facilitating subject‐wide evidence syntheses (Martin et al, 2022; Sutherland & Wordley, 2018).…”

Keeping Meta-Analyses Alive and Well: A Tutorial on Implementing and Using Community-Augmented Meta-Analyses in PsychOpen CAMA

“…Consequently, such outdated syntheses lead to inaccurate and invalid conclusions (Créquit et al, 2016). Addressing these shortcomings, novel, web-based, open-repository concepts for meta-analyses and systematic reviews emerged in various research areas of psychology and neighboring fields (e.g., Bosco et al, 2015; Elliott et al, 2017; LeBel et al, 2018; Shackelford et al, 2021; Spadaro et al, 2022; Tsuji et al, 2014). Some recent applications provide timely syntheses in urgently relevant topics in health research (e.g., Counotte et al, 2018; Michelen et al, 2021) or use the databases for efficiently searching subsets of studies to answer specific research questions in social sciences (e.g., Steel et al, 2018; Yuan et al, 2022).…”