Potential Technologies Review: A hybrid information retrieval framework to accelerate demand‐pull innovation in biomedical engineering

Abstract

Launching biomedical innovations based on clinical demands instead of translating basic research findings to practice reduces the risk that the results will not fit the clinical routine. To realize this type of innovation, a meta‐analysis of the body of research is necessary to reveal demand‐matching concepts. However, both the data deluge and the narrow time constraints for innovation make it impossible to perform such reviews manually. Thus, this paper proposes a specifically adapted “Potential Technologies Review” approach focusing on automated text mining and information retrieval techniques. The novel framework combines features from both systematic and scoping reviews. It aims at high coverage and reproducibility while mapping technologies—even with a fuzzy initial scope. To achieve these goals for search and triage, a set of closely interrelated methods has been developed: (a) automated query optimization, (b) screening prioritization, and (c) recall estimation. To determine appropriate parameters, a variety of published literature corpora were used and compared with an evaluation on a real‐world dataset. Our results show that it is feasible to automate the identification of relevant works using this newly introduced framework. It achieved a workload reduction of up to 91% “Work‐saved‐over Sampling (WSS)” with a 76% overall recall compared with manually screening search results. Reducing the workload is a prerequisite for a rapid Potential Technologies Review when conducting demand‐pull innovations. Moreover, it facilitates the updating and closer monitoring of latest findings. Studying the robustness of the framework and expanding it to patent documents are future tasks.

Automation of systematic reviews of biomedical literature: a scoping review of studies indexed in PubMed

“…The first included paper from 2011 applied text-mining to construct a search syntax for PubMed, using the Apache Lucene platform [ 33 ]. Eleven papers used a plethora of text-mining tools to aid search syntax building, such as Anne O’Tate, AntConc, Apache Lucene, BiblioShiny, Carrot2, CitNetExplorer, EndNote, Keyword‐Analyzer, Leximancer, Lingo3G, Lingo4G, MeSH on Demand, MetaMap, Microsoft Academic, PubReMiner, Systematic Review Accelerator, TerMine, Text Analyzer, Tm for R, VOSviewer, Voyant, Yale MeSH Analyzer, and in-house solutions [ 18 , 33 – 35 , 37 , 41 , 46 , 47 , 49 – 51 ]. Two papers introduced curated article collections, such as Cochrane CENTRAL [ 44 ], and the Realtime Data Synthesis and Analysis (REDASA) COVID-19 dataset [ 48 ], which were assembled using various automation techniques.…”

“…By screening relevant records early, subsequent phases of the SR can advance faster. Other studies applied a combination of strategies [ 41 , 129 ], used alternative methods such as filtering [ 18 ], or similarity of Medline elements [ 130 ], reported the automation software without detailing the strategy [ 131 – 133 ], used convenience tools to speed up screening [ 134 , 135 ], or omitted record screening and applied topic modeling directly to full-text selection [ 45 ].…”

“…As an input to machine learning models, most often bag-of-words (BOW) text representations were applied ( N = 30/89, 33.7%) [ 32 , 41 , 52 , 54 – 56 , 59 , 61 , 68 , 72 , 82 , 84 , 85 , 87 , 89 , 92 , 93 , 95 , 96 , 100 , 106 , 108 , 110 , 112 , 114 , 115 , 119 – 122 ], followed by term-frequency/inverse document frequency (TF-IDF) ( N = 16/89, 18.0%) [ 45 , 53 , 57 , 60 , 63 , 66 , 68 , 73 , 76 , 83 , 91 , 109 , 115 , 116 , 122 , 123 ], topic models ( N = 10/89, 11.2%) [ 45 , 60 , 84 , 86 , 91 , 93 , 104 , 107 , 109 , 115 , 123 ], keywords ( N = 9, 10.1%) [ 52 , 75 , 76 , 91 , 98 , 100 , 117 , 123 , 127 ], standardized terms such as Medical Subject ...…”

“…Few studies used full-text input ( N = 5, 5.6%) [ 45 , 63 , 98 , 125 , 127 ] and database records from ClinicalTrials.gov or Cochrane ( N = 4/89, 4.5%) [ 78 , 79 , 109 , 115 ]. We note that some studies were conducted on published SR databases, such as the EPPI Centre database [ 60 , 83 , 84 , 86 , 89 ] or those from the Oregon Drug Effectiveness Review Project (DERP) [ 32 , 41 , 54 , 68 , 77 , 87 , 88 , 93 , 107 , 120 – 122 , 130 ]. Links to public SR resources were extracted and provided in Additional file 4.…”

“…Some studies combined two stages from either search, screening, full-text selection, or data extraction. These studies included two ASRs [ 45 , 126 ], studies on alternative SR approaches, such as Rapid Evidence Mapping [ 125 ] and Potential Technologies Review [ 41 ], and the REDASA COVID-19 dataset [ 48 ]. A study used automated record screening before evaluating a text mining algorithm for full text selection [ 98 ], and another automated record screening in connection with PICO named entry recognition for data extraction [ 81 ].…”