Statistical Stopping Criteria for Automated Screening in Systematic Reviews

Callaghan, Max; Müller-Hansen, Finn

doi:10.21203/rs.2.18218/v2

Cited by 7 publications

(5 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To enable screening of relevant papers, we applied a novel machine learning algorithm using support vector machines 51 to rank the studies in the order of relevance of their abstracts. A team of four reviewers then manually screened the abstracts of the top 6,023 studies.…”

Section: Methodsmentioning

confidence: 99%

A multi-country meta-analysis on the role of behavioural change in reducing energy consumption and CO2 emissions in residential buildings

et al. 2021

Self Cite

View full text Add to dashboard Cite

Despite the importance of evaluating all mitigation options so as to inform policy decisions addressing climate change, a systematic analysis of household-scale interventions to reduce carbon emissions is missing. Here, we address this gap through a state-of-the-art machine-learning assisted meta-analysis to comparatively assess the effectiveness of a range of monetary and behavioral interventions in energy demand of residential buildings. We identify 122 studies and extract 360 effect sizes representing trials on 1.2 million households in 25 countries. We find that all the studied interventions reduce energy consumption of households. Our meta-regression evidences that monetary incentives are on an average more effective than behavioral interventions, but deploying the right combinations of interventions together can increase overall effectiveness. We estimate global cumulative carbon emissions reduction of 8.64 Gt CO2 by 2040, though deploying the most effective packages and interventions could result in greater reduction. While modest, this potential should be viewed in conjunction with the need for derisking mitigation with energy demand reductions and realizing substantial co-benefits.

show abstract

Section: Methodsmentioning

confidence: 99%

A multi-country meta-analysis on the role of behavioural change in reducing energy consumption and CO2 emissions in residential buildings

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…The initial two batches of documents were screened and coded using the NACSOS platform [41] at the title and abstract level by the whole author team. Differences in opinion were discussed in plenum.…”

Section: Whether It Addressed Policiesmentioning

confidence: 99%

Mapping climate mitigation policy literature using machine learning: disparities between scientific attention, policy density, and emissions

Callaghan,

Banisch,

Doebbeling-Hildebrandt

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Current climate mitigation policies are not sufficient to meet the Paris temperature target, and ramping up efforts will require rapid learning from the scientific literature on climate policies. This literature is vast and widely dispersed, as well as hard to define and categorise, hampering systematic efforts to learn from it. We use a machine learning pipeline using transformer-based language models to systematically map the relevant scientific literature on climate policies at scale and in real-time. Our ``living systematic map'' of climate policy research features a set of 84,990 papers, and classifies each of them by policy instrument type, sector, and geography. We explore how the distribution of these papers varies across countries, and compare this to the distribution of emissions and enacted climate policies. Results suggests a potential stark under-representation of industry sector policies, as well as diverging attention between science and policy with respect to economic and regulatory instruments. Our living map of climate policy research is publicly available and may both usefully support upcoming climate change assessments such as those by the Intergovernmental Panel on Climate Change (IPCC) and serve as a useful resource for the climate policy research community.

show abstract

“…In addition, instead of using just NLP technology, we could employ image processing, useful for detecting graphs, tables and matrices. [33] Fourth, stopping rules are still being developed [34]. Ideally, a researcher would be informed when they have likely passed WSS95%.…”

Section: Future Workmentioning

confidence: 99%

Systematic Review using a Spiral approach with Machine Learning

Saeidmehr

Steel

Samavati

2023

Preprint

View full text Add to dashboard Cite

With the accelerating growth of the academic corpus, doubling every nine years, machine learning is a promising avenue to make systematic review manageable. Though several notable advancements have already been made, the incorporation of machine learning is less than optimal, still relying on a sequential, staged process designed to accommodate a purely human approach, exemplified by PRISMA. Here, we test a spiral, alternating or oscillating approach, where full-text screening is done intermittently with title/abstract screening, which we examine in three datasets by simulation under 360 conditions comprised of different algorithmic classifiers, feature extractions, prioritization rules, data types, and information provided (e.g., title/abstract, full-text included). Overwhelmingly, the results favored a spiral processing approach with Logistic Regression, TF-IDF for vectorization, and Maximum Probability for prioritization. Results demonstrate up to a 90\% improvement over traditional machine learning methodologies, especially for databases with fewer eligible articles. With these advancements, the screening component of most systematic reviews should remain functionally achievable for another one to two decades.

show abstract

Statistical Stopping Criteria for Automated Screening in Systematic Reviews

Cited by 7 publications

References 19 publications

A multi-country meta-analysis on the role of behavioural change in reducing energy consumption and CO2 emissions in residential buildings

A multi-country meta-analysis on the role of behavioural change in reducing energy consumption and CO2 emissions in residential buildings

Mapping climate mitigation policy literature using machine learning: disparities between scientific attention, policy density, and emissions

Systematic Review using a Spiral approach with Machine Learning

Contact Info

Product

Resources

About