SWIFT-Review: a text-mining workbench for systematic review

Howard, Brian E.; Phillips, Jason; Miller, Kyle; Tandon, Arpit; Mav, Deepak; Shah, Mihir; Holmgren, Stephanie; Pelch, Katherine E.; Walker, Vickie R.; Rooney, Andrew A.; Macleod, Malcolm; Shah, Ruchir; Thayer, Kristina A.

doi:10.1186/s13643-016-0263-z

Cited by 147 publications

(150 citation statements)

References 19 publications

Supporting

Mentioning

148

Contrasting

Unclassified

Order By: Relevance

“…The system is interoperable with a related desktop application, SWIFT‐Review, which is freely available. A cross‐validation evaluation across 20 previously completed reviews, including 15 from Cohen et al, has shown consistent work saved over sampling.…”

Section: Related Workmentioning

confidence: 99%

“…References with the same topic or cluster may be thematically related even if there are no common keywords they all share. Furthermore, the topic proportions of each reference can be used to find related references, and as a feature representation for machine learning . Primarily, clusters and topics, supplemented with automatically generated descriptions, allow reviewers to explore the thematic coverage and locate relevant references, without having to explicitly form keyword queries, within diverse collections.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prioritising references for systematic reviews with RobotAnalyst: A user study

Przybyła

Brockmeier

Kontonatsios

et al. 2018

Research Synthesis Methods

108

View full text Add to dashboard Cite

Screening references is a time‐consuming step necessary for systematic reviews and guideline development. Previous studies have shown that human effort can be reduced by using machine learning software to prioritise large reference collections such that most of the relevant references are identified before screening is completed. We describe and evaluate RobotAnalyst, a Web‐based software system that combines text‐mining and machine learning algorithms for organising references by their content and actively prioritising them based on a relevancy classification model trained and updated throughout the process. We report an evaluation over 22 reference collections (most are related to public health topics) screened using RobotAnalyst with a total of 43 610 abstract‐level decisions. The number of references that needed to be screened to identify 95% of the abstract‐level inclusions for the evidence review was reduced on 19 of the 22 collections. Significant gains over random sampling were achieved for all reviews conducted with active prioritisation, as compared with only two of five when prioritisation was not used. RobotAnalyst's descriptive clustering and topic modelling functionalities were also evaluated by public health analysts. Descriptive clustering provided more coherent organisation than topic modelling, and the content of the clusters was apparent to the users across a varying number of clusters. This is the first large‐scale study using technology‐assisted screening to perform new reviews, and the positive results provide empirical evidence that RobotAnalyst can accelerate the identification of relevant studies. The results also highlight the issue of user complacency and the need for a stopping criterion to realise the work savings.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Prioritising references for systematic reviews with RobotAnalyst: A user study

Przybyła

Brockmeier

Kontonatsios

et al. 2018

Research Synthesis Methods

108

View full text Add to dashboard Cite

show abstract

“…Our review, which required major investment in time and human capital, ultimately only yielded a small number of relevant studies even as initial searches suggested more than 50,000 might be relevant. Machine learning and other technological methods are increasingly available to aid in systematic reviews at the title and abstract and full text stages [63] and other approaches are now available to help use a broader range of datasets (e.g. Google Scholar) [64].…”

Section: Review Limitationsmentioning

confidence: 99%

What are the environmental impacts of property rights regimes in forests, fisheries and rangelands?

et al. 2017

View full text Add to dashboard Cite

Background: Property rights to natural resources comprise a major policy instrument in efforts to advance sustainable resource use and conservation. Debate over the relative effectiveness of different property rights regimes in reaching these goals remains controversial. A large, diverse, and rapidly growing body of literature investigates the links between property rights regimes and environmental outcomes, but has not synthesized theoretical and policy insights within specific resource systems and especially across resource systems. Methods:We conducted a systematic review following CEE Guidelines in which we collected empirical evidence from the past two decades on the environmental impacts of property rights regimes in fisheries, forests and rangelands in developing countries. We used a bundle of rights approach to assess the impacts of state, private, and community property regimes, as well as mixed regimes and open access conditions. Outcomes were classified as positive, negative, neutral or undetermined. We also collected information on contextual and other factors thought to influence effect of property rights regimes on environmental outcomes. The search covered 90 online databases and three languages, resulting in a total of 34,984 screened titles.Results: This review identified 103 articles consisting of 374 property regime studies: 55% of the studies related to forestry, 31% to fisheries, and 14% to rangelands. The majority of the studies comprised case-control studies but presented limited information on the baseline condition of the resource system. Only 26 studies used before-aftercontrol-impact (BACI) design. We found that property regime comparisons differed across resource domains with, for example, the majority of fisheries studies using comparison to an open access situation while forest and rangeland studies were more mixed in regime comparisons. After critical appraisal of included studies, only 80 studies were accepted for the narrative synthesis. The key contextual factors largely associated with reported positive environmental outcomes across the three resource systems included monitoring and enforcement systems, resource use pressure, and the presence or absence of clear, stable and legitimate rights (i.e. 'positive regime characteristics').

show abstract

“…For example, SWIFT-review provides a workbench with tools to assist the reviewer in literature prioritisation (Howard et al, 2016). Using these tools, the reviewer can determine which articles to screen first, thereby reducing the amount of time spent on irrelevant papers.…”

Section: Retrieval and Screening Stage Of A Systematic Reviewmentioning

confidence: 99%

Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA

Jaspers

Troyer

Aerts

2018

EFS3

View full text Add to dashboard Cite

This Report presents the results from EFSA project RC/EFSA/AMU/2016/01 related to the implementation of machine learning techniques for literature reviews and systematic reviews in EFSA. An overview of the different steps of a systematic review is provided, along with possible ways for automation. Although it was found that most steps could benefit from automation, it was also observed that some steps require more sophisticated methods than those encompassed within the machine learning framework. Availability of data and methodology allowed for the development of an automatic screening tool based on several machine learning techniques. The developed shiny R application can be used for the screening of abstracts and full texts. Properties of machine learning techniques are discussed in this Report together with their most important advantages and disadvantages. The latter discussion includes both general properties, as well as context-specific properties based on their performance in three case studies. Although creating a universal automatic data extraction tool was considered to be infeasible in this stage, this step of the systematic review was addressed to allow the reviewer to scan the uploaded pdf files for certain words or string of words. Based on observations from the performed case studies, recommendations were made regarding which methods are preferred in specific situations. More explicitly, a discussion is made about the performance of the classifiers with respect to the magnitude of the pool of papers to be screened as well as to the amount of imbalance, referring to the proportion of relevant and irrelevant papers. Finally, it was concluded that the results presented in this report provide proof that the developed shiny application could be efficiently used in combination with other software such as DistillerSR. © European Food Safety Authority, 2018Key words: Systematic Reviews, Machine Learning, screening, data extraction, Sensitivity, Specificity Disclaimer: The present document has been produced and adopted by the bodies identified above as author(s). This task has been carried out exclusively by the author(s) in the context of a contract between the European Food Safety Authority and the author(s), awarded following a tender procedure. The present document is published complying with the transparency principle to which the Authority is subject. It may not be considered as an output adopted by the Authority. The European Food Safety Authority reserves its rights, view and position as regards the issues addressed and the conclusions reached in the present document, without prejudice to the rights of the authors. Reproduction is authorised provided the source is acknowledged. Machine Learning Techniques for Literature and Systematic Reviewswww.efsa.europa.eu/publications 3 EFSA Supporting publication 2018:EN-1427The present document has been produced and adopted by the bodies identified above as author. This task has been carried out exclusively by the author in the context of a contract ...

show abstract

SWIFT-Review: a text-mining workbench for systematic review

Cited by 147 publications

References 19 publications

Prioritising references for systematic reviews with RobotAnalyst: A user study

Prioritising references for systematic reviews with RobotAnalyst: A user study

What are the environmental impacts of property rights regimes in forests, fisheries and rangelands?

Machine learning techniques for the automation of literature reviews and systematic reviews in EFSA

Contact Info

Product

Resources

About