Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study

Lefebvre, Carol A.; Glanville, Julie; Beale, Sophie; Boachie, Charles; Duffy, Steven; Fraser, Cynthia; Harbour, Jenny; McCool, Rachael; Smith, Lynne

doi:10.3310/hta21690

Cited by 36 publications

(41 citation statements)

References 117 publications

(346 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While combining these methods is recommended [9], , it may not eliminate bias caused by any one method. Accordingly, ur gold standard sample may not be fully representative of those in the corpus of literature, particularly studies where the inequality is not indexed and reported only in the body of the text.…”

Section: Discussionmentioning

confidence: 99%

“…However, studies contained in the reviews we examined may not be representative of the full corpus of literature and important studies may be missed [5]. Further, the reviews we located may not themselves fully represent the sample universe of equity-focused reviews [9]. We hand-searched only a limited number of issues in two journals, and records contained in personal files are unlikely to have broad coverage of the topic area.…”

Section: Discussionmentioning

confidence: 99%

“…We used the same method to develop and measure the performance of our sample (internal validation). Validation of an external set, located by different methods than those used for the evaluation, would have strengthened our study [9] but this was beyond current resources. Although we did not formally assess reliability, two authors experienced in social epidemiology independently identified gold standard studies.…”

Section: Discussionmentioning

confidence: 99%

“…Further development could include refining to improve sensitivity and specificity, screening all retrieved records for equity studies in order to calculate accuracy and precision, and validating against an external standard [9]. …”

Section: Discussionmentioning

confidence: 99%

“…Our validation method is considered an internal gold standard [9]. In order to examine indexing, we also calculated sensitivity for selected subject headings we considered to be general to the social determinants of health.…”

Section: Methodsmentioning

confidence: 99%

See 4 more Smart Citations

Development and validation of a search filter to identify equity-focused studies: reducing the number needed to screen

Prady

Uphoff

Power

et al. 2018

BMC Med Res Methodol

View full text Add to dashboard Cite

BackgroundHealth inequalities, worse health associated with social and economic disadvantage, are reported by a minority of research articles. Locating these studies when conducting an equity-focused systematic review is challenging due to a deficit in standardised terminology, indexing, and lack of validated search filters. Current reporting guidelines recommend not applying filters, meaning that increased resources are needed at the screening stage.MethodsWe aimed to design and test search filters to locate studies that reported outcomes by a social determinant of health. We developed and expanded a ‘specific terms strategy’ using keywords and subject headings compiled from recent systematic reviews that applied an equity filter. A ‘non-specific strategy’ was compiled from phrases used to describe equity analyses that were reported in titles and abstracts, and related subject headings. Gold standard evaluation and validation sets were compiled. The filters were developed in MEDLINE, adapted for Embase and tested in both. We set a target of 0.90 sensitivity (95% CI; 0.84, 0.94) in retrieving 150 gold standard validation papers. We noted the reduction in the number needed to screen in a proposed equity-focused systematic review and the proportion of equity-focused reviews we assessed in the project that applied an equity filter to their search strategy.ResultsThe specific terms strategy filtered out 93-95% of all records, and retrieved a validation set of articles with a sensitivity of 0.84 in MEDLINE (0.77, 0.89), and 0.87 (0.81, 0.92) in Embase. When combined (Boolean ‘OR’) with the non-specific strategy sensitivity was 0.92 (0.86, 0.96) in MEDLINE (Embase 0.94; 0.89, 0.97). The number needed to screen was reduced by 77% by applying the specific terms strategy, and by 59.7% (MEDLINE) and 63.5% (Embase) by applying the combined strategy. Eighty-one per cent of systematic reviews filtered studies by equity.ConclusionsA combined approach of using specific and non-specific terms is recommended if systematic reviewers wish to filter studies for reporting outcomes by social determinants. Future research should concentrate on the indexing standardisation for equity studies and further development and testing of both specific and non-specific terms for accurate study retrieval.Electronic supplementary materialThe online version of this article (10.1186/s12874-018-0567-x) contains supplementary material, which is available to authorized users.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

See 3 more Smart Citations