Text mining for search term development in systematic reviewing: A discussion of some methods and challenges

Abstract

Using text mining to aid the development of database search strings for topics described by diverse terminology has potential benefits for systematic reviews; however, methods and tools for accomplishing this are poorly covered in the research methods literature. We briefly review the literature on applications of text mining for search term development for systematic reviewing. We found that the tools can be used in five overarching ways: improving the precision of searches; identifying search terms to improve search sensitivity; aiding the translation of search strategies across databases; searching and screening within an integrated system; and developing objectively-derived search strategies. Using a case study and selected examples, we then reflect on the utility of certain technologies (TF-IDF and Termine, term frequency, and clustering) in improving the precision and sensitivity of searches. Challenges in using these tools are discussed. The utility of these tools is influenced by the different capabilities of the tools, the way the tools are used, and the text that is analysed. Increased awareness of how the tools perform facilitates the further development of methods for their use in systematic reviews.

Potential Technologies Review: A hybrid information retrieval framework to accelerate demand‐pull innovation in biomedical engineering

“…Starting from a relatively small, manually created corpus of relevant work matching the given clinical demand, a complex query is automatically generated from terms including Boolean operators to expand the search and then the recall‐optimized version is selected in an iterative comparison with PubMed. This procedure does not only go far beyond the ATR approaches, eg, RAKE or Termine used so far in systematic reviews, but in combination with the query‐based prioritization indicates that it does not tend towards “hasty generalization,” but on the contrary also includes new aspects at the edges of the topic area. This has to be investigated in more detail in the future.…”

“…In addition, user feedback can be implemented to control the term extraction by an intuitive interface 32 or by providing an initial corpus of a small number of manually assessed relevant citations. 33 Although many text mining technologies are analyzed to accelerate review, almost all studies focus explicitly on systematic rather than scoping reviews. Among the few exceptions, eg, Stansfield et al work on clustering topics following the screening phase, 34 the study of Shemilt et al 10 clearly stands out for many reasons.…”

“…Consequently, automatic term recognition (ATR) is conducted to identify concepts of a corpus represented by terms in a weighted order either to refine the search strategy or to prioritize the screening. In addition, user feedback can be implemented to control the term extraction by an intuitive interface or by providing an initial corpus of a small number of manually assessed relevant citations …”

Approaches to evidence synthesis in international development: a research agenda

“…The potential of automation can be considered in two broad areas: making parts of the existing systematic review process more efficient; and changing the way in which research evidence is organised in more 'upstream' ways In terms of making existing methods more efficient, new technologies are being developed which assist at each stage of the review process, with those assisting in the earlier stages more mature than those in the later parts. When developing search strategies, text mining and natural language processing technologies can be helpful in increasing both the precision and sensitivity of searches and in 'translating' search strategies across different data sources (Stansfield et al 2017). Given the time-consuming and repetitive nature of manual citation 'screening' (the activity of sifting through the -often -thousands of irrelevant research records in order to find the relatively few that match the review's inclusion criteria), much effort has gone into automating the identification of studies and reducing the human resource required for this activity.…”

Evidence surveillance to keep up to date with new research

“…Previous work has shown that the way that search strategies are developed can impact on the recall of the search (e.g. [6]), and in a case-study, Stansfield and colleagues [7] describe how text analytic software is able to assist in five ways:…”