Can abstract screening workload be reduced using text mining? User experiences of the tool Rayyan

Olofsson, Hanna; Brolund, Agneta; Hellberg, Christel; Silverstein, Rebecca A.; Stenström, Karin; Österberg, Marie; Dagerhamn, Jessica

doi:10.1002/jrsm.1237

Cited by 75 publications

(78 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Classic systematic review and meta‐analysis textbooks also detail the basics of the abstract screening process; yet, many textbooks lack practical information about best practice methods for this stage such as the use of text‐mining software, pilot training of coders, reconciliation, or managing screeners. Other more recent research focuses on improving the efficiency of screening abstracts by evaluating the effectiveness of text‐mining software . These studies focus on the efficiency of the algorithms used for text‐mining but fail to provide practical advice on how review teams can implement text‐mining software in large‐evidence systematic review projects.…”

Section: Introductionmentioning

confidence: 99%

“…Other more recent research focuses on improving the efficiency of screening abstracts by evaluating the effectiveness of text-mining software. [14][15][16] These studies focus on the efficiency of the algorithms used for text-mining but fail to provide practical advice on how review teams can implement text-mining software in large-evidence systematic review projects.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses

Polanin

Pigott

Espelage

et al. 2019

Research Synthesis Methods

206

126

View full text Add to dashboard Cite

screening is one important aspect of conducting a high‐quality and comprehensive systematic review and meta‐analysis. Abstract screening allows the review team to conduct the tedious but vital first step to synthesize the extant literature: winnowing down the overwhelming amalgamation of citations discovered through research databases to the citations that should be “full‐text” screened and eventually included in the review. Although it is a critical process, few guidelines have been put forth since the publications of seminal systematic review textbooks. The purpose of this paper, therefore, is to provide a practical set of best practice guidelines to help future review teams and managers. Each of the 10 proposed guidelines is explained using real‐world examples or illustrations from applications. We also delineate recent experiences where a team of abstract screeners double‐screened 14 923 abstracts in 89 days.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses

Polanin

Pigott

Espelage

et al. 2019

Research Synthesis Methods

206

126

View full text Add to dashboard Cite

show abstract

“…Stringent requirements exist for the transparency of the study selection process and the reliability of the corresponding results. These requirements aim to avoid the non-detection of relevant evidence with a subsequent risk of bias endangering the validity of conclusions based on the available evidence [ 1 , 2 ].…”

Section: Introductionmentioning

confidence: 99%

“…The double-screening approach has the following advantages: firstly, it can be ensured that the study inclusion criteria are applied consistently, thus avoiding systematic errors, and secondly, random errors such as careless mistakes can be identified and corrected. However, the approach is resource intensive, which can be a problem, as systematic reviews generally need to be completed within a defined period with a limited budget [ 1 , 2 ].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Effective study selection using text mining or a single-screening approach: a study protocol

et al. 2018

View full text Add to dashboard Cite

BackgroundSystematic information retrieval generally requires a two-step selection process for studies, which is conducted by two persons independently of one another (double-screening approach). To increase efficiency, two methods seem promising, which will be tested in the planned study: the use of text mining to prioritize search results as well as the involvement of only one person in the study selection process (single-screening approach). The aim of the present study is to examine the following questions related to the process of study selection: Can the use of the Rayyan or EPPI Reviewer tools to prioritize the results of study selection increase efficiency? How accurately does a single-screening approach identify relevant studies? Which advantages or disadvantages (e.g., shortened screening time or increase in the number of full texts ordered) does a single-screening versus a double-screening approach have?MethodsOur study is a prospective analysis of study selection processes based on benefit assessments of drug and non-drug interventions. It consists of two parts: firstly, the evaluation of a single-screening approach based on a sample size calculation (11 study selection processes, including 33 single screenings) and involving different screening tools and, secondly, the evaluation of the conventional double-screening approach based on five conventional study selection processes. In addition, the advantages and disadvantages of the single-screening versus the double-screening approach with regard to the outcomes “number of full texts ordered” and “time required for study selection” are analyzed. The previous work experience of the screeners is considered as a potential effect modifier.DiscussionNo study comparing the features of prioritization tools is currently available. Our study can thus contribute to filling this evidence gap. This study is also the first to investigate a range of questions surrounding the screening process and to include an a priori sample size calculation, thus enabling statistical conclusions. In addition, the impact of missing studies on the conclusion of a benefit assessment is calculated.Systematic review registrationNot applicable

show abstract

Machine learning for accelerating screening in evidence reviews

Chappell

Edwards

Watkins

et al. 2023

Cochrane Evidence Synthesis and Methods

View full text Add to dashboard Cite

Evidence reviews are important for informing decision-making and primary research, but they can be time-consuming and costly. With the advent of artificial intelligence, including machine learning, there is an opportunity to accelerate the review process at many stages, with study screening identified as a prime candidate for assistance.Despite the availability of a large number of tools promising to assist with study screening, these are not consistently used in practice and there is skepticism about their application. Single-arm evaluations suggest the potential for tools to reduce screening burden. However, their integration into practice may need further investigation through evaluations of outcomes such as overall resource use and impact on review findings and recommendations. Because the literature lacks comparative studies, it is not currently possible to determine their relative accuracy.In this commentary, we outline the published research and discuss options for incorporating tools into the review workflow, considering the needs and requirements of different types of review.

show abstract

Can abstract screening workload be reduced using text mining? User experiences of the tool Rayyan

Abstract: The text mining function in Rayyan successfully helped reviewers identify relevant studies early in the screening process.

Cited by 75 publications

References 8 publications

Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses

Best practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses

Effective study selection using text mining or a single-screening approach: a study protocol

Machine learning for accelerating screening in evidence reviews

Contact Info

Product

Resources

About