Grey literature: An important resource in systematic reviews

Abstract

Systematic reviews aid the analysis and dissemination of evidence, using rigorous and transparent methods to generate empirically attained answers to focused research questions. Identifying all evidence relevant to the research questions is an essential component, and challenge, of systematic reviews. Grey literature, or evidence not published in commercial publications, can make important contributions to a systematic review. Grey literature can include academic papers, including theses and dissertations, research and committee reports, government reports, conference papers, and ongoing research, among others. It may provide data not found within commercially published literature, providing an important forum for disseminating studies with null or negative results that might not otherwise be disseminated. Grey literature may thusly reduce publication bias, increase reviews' comprehensiveness and timeliness and foster a balanced picture of available evidence. Grey literature's diverse formats and audiences can present a significant challenge in a systematic search for evidence. However, the benefits of including grey literature may far outweigh the cost in time and resource needed to search for it, and it is important for it to be included in a systematic review or review of evidence. A carefully thought out grey literature search strategy may be an invaluable component of a systematic review. This narrative review provides guidance about the benefits of including grey literature in a systematic review, and sources for searching through grey literature. An illustrative example of a search for evidence within grey literature sources is presented to highlight the potential contributions of such a search to a systematic review. Benefits and challenges of grey literature search methods are discussed, and recommendations made.

Barriers and enablers to accessing dental services for people experiencing homelessness: A systematic review

“…Furthermore, thematic analysis is a tested method that provides an explicit link between the primary studies and the conclusions drawn and thus adheres to principles that are important in systematic reviews . The inclusion of grey literature reduced the possibility of publication bias and allowed a more balanced view of the existing evidence …”

“…31 The inclusion of grey literature reduced the possibility of publication bias and allowed a more balanced view of the existing evidence. 58 The model developed could be used in a variety of settings to explore avenues for successful implementation of oral health promotion services for people who are homeless. Another strength of this review lies in the fact that both the patients and providers/carers' perspective were explored, and thus, bias towards either the individual or structural components of access was limited.…”

A Challenge for Clinical Pharmacologists: How Can We Measure Scientific Impact of Publications in Drug Development and in Regulatory Decision Making?

“…Nonetheless, publication of articles in journals with a high JIF is deemed (implicitly and explicitly) to indicate the value of scholarly work and contributes to tenure and funding decisions. However, the JIF does not account for journals not included in the Journal Citation Reports, nor does it account for “grey” content (i.e., evidence not appearing in commercial publications), including: legal briefs, theses and dissertations, government reports, patent applications, expert testimony, drug development, and regulatory decision‐making documents. In response to an evolving scientific and professional communications landscape, there are new fields focused on investigating and developing nontraditional, objective measures of scientific impact, including the Relative Citation Ratio by the National Institutes of Health, CiteScore by Elsevier, and Impact Quotient, none of which have been embraced by the scientific community.…”

“…The appropriateness of including nontraditional sources also depends on the vetting process of the information. Clearly, a report where a scientific panel was used to vet all data may be as important as a published article; unpublished negative data may also provide critical information . One potential metric is the number of citations in regulatory documents (e.g., briefing documents, new drug applications, and investigational drug applications).…”

“…Clearly, a report where a scientific panel was used to vet all data may be as important as a published article; unpublished negative data may also provide critical information. 2 One potential metric is the number of citations in regulatory documents (e.g., briefing documents, new drug applications, and investigational drug applications). However, because these are confidential documents, the citations are not currently available for inspection by the scientific community.…”