Development and validation of paired MEDLINE and Embase search filters for cost-utility studies

Abstract

Background Search filters are standardised sets of search terms, with validated performance, that are designed to retrieve studies with specific characteristics. A cost–utility analysis (CUA) is the preferred type of economic evaluation to underpin decision-making at the National Institute for Health and Care Excellence (NICE). Until now, when searching for economic evidence for NICE guidelines, we have used a broad set of health economic-related search terms, even when the reviewer’s interest is confined to CUAs alone. METHODS: We developed search filters to retrieve CUAs from MEDLINE and Embase. Our aim was to achieve recall of 90% or better across both databases while reducing the overall yield compared with our existing broad economic filter. We used the relative recall method along with topic expert input to derive and validate 3 pairs of filters, assessed by their ability to identify a gold-standard set of CUAs that had been used in published NICE guidelines. We developed and validated MEDLINE and Embase filters in pairs (testing whether, when used together, they find target studies in at least 1 database), as this is how they are used in practice. We examined the proxy-precision of our new filters by comparing their overall yield with our previous approach using publications indexed in a randomly selected year (2010). RESULTS: All 3 filter-pairs exceeded our target recall and led to substantial improvements in search proxy-precision. Our paired ‘sensitive’ filters achieved 100% recall (95% CI 99.0 to 100%) in the validation set. Our paired ‘precise’ filters also had very good recall (97.6% [95%CI: 95.4 to 98.9%]). We estimate that, compared with our previous search strategy, using the paired ‘sensitive’ filters would reduce reviewer screening burden by a factor of 5 and the ‘precise’ versions would do so by a factor of more than 20. CONCLUSIONS: Each of the 3 paired cost–utility filters enable the identification of almost all CUAs from MEDLINE and Embase from the validation set, with substantial savings in screening workload compared to our previous search practice. We would encourage other researchers who regularly use multiple databases to consider validating search filters in combination as this will better reflect how they use databases in their everyday work.

Emergency department crowding: an overview of reviews describing measures causes, and harms

“…A health sciences librarian with experience in systematic reviews (HG) searched MEDLINE and Embase on the Ovid platform, from inception to October 5, 2022, for subject headings and keywords related to the concepts of emergency department and overcrowding. A search filter for systematic reviews, originally developed by the Canadian Agency for Drugs and Technologies in Health [ 16 ] was slightly modified to include scoping reviews and applied to the base search. Complete search strategies can be found in Appendix 1.…”

Diagnostics and treatments of COVID-19: two-year update to a living systematic review of economic evaluations

“…This update to our “living” systematic review is methodologically consistent with the first publication ( Elvidge et al, 2022 ), and so the same issues concerning the search strategy and generalisability of findings apply here. Our review aimed to provide a comprehensive account of available evidence and as such, a large number of unique records were identified by the database search (8,287); this is due to the known sensitivity of search terms used to identify economic evaluations ( Hubbard et al, 2022 ). This increases the sensitivity of the search, reducing the likelihood of missing relevant studies, but it also means future updates will continue to require a labour-intensive screening process.…”

Development and validation of a geographic search filter for MEDLINE (PubMed) to identify studies conducted in Germany

“…First, the relative recall approach employed to create the GSS may introduce a potential limitation, as it could bias the developed filter towards the search terms identified in the GSS (38). However, in the case of our filter, this limitation is unlikely, given that our GSS included records related to primary studies covering a diverse range of topics and study types, spanning almost 30 years of publication.…”