Although there are now several bibliographic databases of research publications, such as Google Scholar, Pubmed, Scopus, and the Web of Science (WoS), and some also include counts of citations, there is at present no similarly comprehensive database of the rapidly growing number of clinical practice guidelines (CPGs), with their references, which sometimes number in the hundreds. CPGs have been shown to be useful for the evaluation of clinical (as opposed to basic) biomedical research, which often suffers from relatively low counts of citations in the serial literature. The objectives were to introduce a new citation database, clinical impact ® , and demonstrate how it can be used to evaluate research impact of clinical research publications by exploring the characteristics of CPG citations of two sets of papers, as well as show temporal variation of clinical impact ® and the WoS. The paper includes the methodology used to retain the data and also the rationale adopted to achieve data quality. The analysis showed that although CPGs tend preferentially to cite papers from their own country, this is not always the case. It also showed that cited papers tend to have a more clinical research level than uncited papers. An analysis of diachronous citations in both clinical impact ® and the WoS showed that although the WoS citations showed a decreasing trend after a peak at 2-3 years after publication, this was less clear for CPG citations and a longer timescale would be needed to evaluate their impact on these documents.

Background: The impact of medical research is usually judged on the basis of citations in the serial literature. A better test of its utility is through its contribution to clinical practice guidelines (CPGs) on how to prevent, diagnose, and treat illness. This study aimed to compare the parameters of lung cancer research papers with those cited as references in lung cancer CPGs from 16 countries, and the Cochrane Collaboration. These comparisons were mainly based on bibliographic data compiled from the Web of Science (WoS). Methodology: We examined 7357 references (of which 4491 were unique) cited in a total of 77 lung cancer CPGs, and compared them with 73,214 lung cancer papers published in the WoS between 2004 and 2018. Results: References used by lung CPGs were much more clinical than the overall body of research papers on this cancer, and their authors predominantly came from smaller northern European countries. However, the leading institutions whose papers were cited the most on these CPGs were from the USA, notably the MD Anderson Cancer Center in Texas, the Memorial Sloan Kettering Cancer Center, New York, and the Mayo Clinic in Rochester, Minnesota. The types of research cited by the CPGs were primarily clinical trials, as well as three treatment modalities (chemotherapy, radiotherapy and surgery). Genetics, palliative care and quality of life were largely neglected. The median time gap between papers cited on a lung CPG and its publication was 3.5 years longer than for WoS citations. Conclusions: Analysis of the references on CPGs allows an alternative means of research evaluation, and one that may be more appropriate for clinical research than citations in academic journals. Own-country references show the direct contribution of research to a country's health care, and other-country references show the esteem in which this research has been held internationally.