Literature searches in systematic reviews and meta-analyses: A review, evaluation, and recommendations

Abstract

The search strategy used in systematic reviews is an important consideration, as the comprehensiveness and representativeness of studies identified influences the quality of conclusions derived from the review. Despite the importance of this step, little in the way of best practice recommendations exist. In an effort to inform future reviews, we report the results of two studies. In Study 1, we outline a series of recommendations for designing comprehensive systematic literature searches. We review the search strategies used in 152 recent systematic reviews published in top applied psychology (including organizational psychology, counseling psychology, and management) journals and evaluate them against these criteria. In Study 2, we build on the findings of our review, carrying out an original meta-analysis, which we use as an opportunity to empirically demonstrate effects of database selection and inclusion of a complementary search protocol on search outcomes. Our results suggest that database selection could have a large effect on conclusions from reviews. Implications and recommendations for carrying out comprehensive literature reviews are described.