Cross-sectional analysis of bibliometrics and altmetrics: comparing the impact of qualitative and quantitative articles in the <i>British Medical Journal</i>

Retrouvey, Helene; Webster, Fiona; Zhong, Toni; Gagliardi, Anna R.; Baxter, Nancy N.

doi:10.1136/bmjopen-2020-040950

Cited by 18 publications

(13 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The median AAS of all included reviews was 17, and was achieved within 3-9 months, which is similar to the median AAS score of 16 for an article in the BMJ achieved after 2 years. 31 Our included reviews had a mean citation of 82 on Google Scholar after a maximum of 9 months, which is substantially higher than the average 12-month Google Scholar citation of 25, for health and medical science articles. 32 Although few had been cited in policy documents, it was concerning that quality of review did not appear to influence this.…”

Section: Evidence Synthesismentioning

confidence: 81%

Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

Abbott

Bethel

Rogers

et al. 2021

BMJ EBM

View full text Add to dashboard Cite

ObjectiveThe academic and scientific community has reacted at pace to gather evidence to help and inform about COVID-19. Concerns have been raised about the quality of this evidence. The aim of this review was to map the nature, scope and quality of evidence syntheses on COVID-19 and to explore the relationship between review quality and the extent of researcher, policy and media interest.Design and settingA meta-research: systematic review of reviews.Information sourcesPubMed, Epistemonikos COVID-19 evidence, the Cochrane Library of Systematic Reviews, the Cochrane COVID-19 Study Register, EMBASE, CINAHL, Web of Science Core Collection and the WHO COVID-19 database, searched between 10 June 2020 and 15 June 2020.Eligibility criteriaAny peer-reviewed article reported as a systematic review, rapid review, overview, meta-analysis or qualitative evidence synthesis in the title or abstract addressing a research question relating to COVID-19. Articles described as meta-analyses but not undertaken as part of a systematic or rapid review were excluded.Study selection and data extractionAbstract and full text screening were undertaken by two independent reviewers. Descriptive information on review type, purpose, population, size, citation and attention metrics were extracted along with whether the review met the definition of a systematic review according to six key methodological criteria. For those meeting all criteria, additional data on methods and publication metrics were extracted.Risk of biasFor articles meeting all six criteria required to meet the definition of a systematic review, AMSTAR-2 ((A MeaSurement Tool to Assess systematic Reviews, version 2.0) was used to assess the quality of the reported methods.Results2334 articles were screened, resulting in 280 reviews being included: 232 systematic reviews, 46 rapid reviews and 2 overviews. Less than half reported undertaking critical appraisal and a third had no reproducible search strategy. There was considerable overlap in topics, with discordant findings. Eighty-eight of the 280 reviews met all six systematic review criteria. Of these, just 3 were rated as of moderate or high quality on AMSTAR-2, with the majority having critical flaws: only a third reported registering a protocol, and less than one in five searched named COVID-19 databases. Review conduct and publication were rapid, with 52 of the 88 systematic reviews reported as being conducted within 3 weeks, and a half published within 3 weeks of submission. Researcher and media interest, as measured by altmetrics and citations, was high, and was not correlated with quality.DiscussionThis meta-research of early published COVID-19 evidence syntheses found low-quality reviews being published at pace, often with short publication turnarounds. Despite being of low quality and many lacking robust methods, the reviews received substantial attention across both academic and public platforms, and the attention was not related to the quality of review methods.InterpretationFlaws in systematic review methods limit the validity of a review and the generalisability of its findings. Yet, by being reported as ‘systematic reviews’, many readers may well regard them as high-quality evidence, irrespective of the actual methods undertaken. The challenge especially in times such as this pandemic is to provide indications of trustworthiness in evidence that is available in ‘real time’.PROSPERO registration numberCRD42020188822.

show abstract

Section: Evidence Synthesismentioning

confidence: 81%

Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

Abbott

Bethel

Rogers

et al. 2021

BMJ EBM

View full text Add to dashboard Cite

show abstract

“…Nevertheless, there was a strong correlation between journals and well-established Twitter accounts (Chang et al, 2019). The same is true for the urological literature (Nocera et al, 2019) and dermatology journal articles (Nip and Feng, 2020) and top 100 cited papers highly cited articles in biology (Zhang and Wang, 2018) and top cited papers in WoS (Retrouvey et al, 2020). As a result, all these studies clarify the fact that neither bibliographic indicators nor altmetric indicators alone can be a complete tool for assessing the dissemination and scientific impact of research.…”

Section: Literature Reviewmentioning

confidence: 85%

“…Despite all that has been said, this study has some limitations. Although we used bibliometric and altmetric indicators to indicate the correlation between academic and social media indicators in articles, according to Retrouvey et al "many of the measures generated by the websites do not have reference values to help adjudicate what is a high or low score" (Retrouvey et al, 2020). Furthermore, altmetric indicators, even along with bibliographic criteria, cannot well represent the ability of their owners to be used to allocate grants or determine the competitive advantage of researchers.…”

Section: Discussionmentioning

confidence: 99%

Relationship between altmetric and bibliometric indicators across academic social sites in article-level: the case of Iranian researchers' “Top Papers” in clinical medicine

Batooli

Mohamadloo

Nadi-Ravandi

2021

LHT

View full text Add to dashboard Cite

PurposeThe study aimed to measure scientific and social impacts of Iranian researchers' “Top Papers” in clinical medicine using citation and altmetric indicators.Design/methodology/approachIn this applied descriptive-analytical study, it used scientometric analysis. A total of 166 “Top Papers” of Iranian researchers in clinical medicine category of Web of Science (WoS) database including “Highly Cited Papers” and “Hot Papers” published between 2009 and 2019 were used. Overall, 29 indicators and their data were extracted from WoS, Scopus, ResearchGate (RG) and PlumX in March 2020.FindingsThe results showed that there exists a positive correlation between the number of citations in WoS, Scopus, RG, PubMed and Crossref. In addition, it was found that there existed a positive correlation between the received citations by articles and altmetric indicators. According to the results, there is a strong correlation between RG Research Interest and citation impact. The correlation analysis on the Plum Analytics categories including “Usage”, “Capture”, “Mention”, “Social Media” and “Citation” showed the correlations between five dimensions of impact were positive and significant. The results have led the authors to think more about new metrics that can response to new developments in the new information areas.Research limitations/implicationsThere are limitations to access altmetric.com in Iran and cannot be used easily. On the other hand, because of considering 24 indicators, authors had to investigate only a sample of 166 top papers from Iranian researchers to present the detailed results. About nature of altmetric indicators, although they reflect the nonacademic impact of articles alongside bibliographic indicators, they still cannot be a complete representative of the influence of their owners.Practical implicationsThis study can indicate a practical application appropriate for the future study. It would be valuable to further examine how social academic platforms construct images of impact of research and how this impacts the social impact of the university as a mission. This study suggests that social media attention to academic research can be much greater than what is shown in traditional indicators such as citation.Originality/valueThis study examines 29 indicators from four platforms including RG, WoS, Scopus and PlumX, simultaneously and measures the relationship among them.

show abstract

“…Bibliometric analysis measures the impact of articles using metrics such as the JIF (JIF) and number of citations [1]. A bibliometric analysis can serve to identify in uential articles that have shaped medical practice, policy decisions, and presented new research ideas [2].…”

Section: Introductionmentioning

confidence: 99%

Bibliometric study of ‘overviews of systematic reviews’ of health interventions: evaluation of prevalence, citation and impact factor

Lunny

Neelakant

Chen

et al. 2021

Preprint

View full text Add to dashboard Cite

Background: Overviews synthesising the results of multiple systematic reviews help inform evidence-based clinical practice. In this first of two companion papers, we evaluate the bibliometrics of overviews, including their prevalence and factors affecting citation rates and JIF (JIF).Methods: We searched MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews (CDSR). We included overviews that: (a) synthesised reviews, (b) conducted a systematic search, (c) had a methods section, and (d) examined a healthcare intervention. Multivariable regression was conducted to determine the association between citation density, JIF and 6 predictor variables. Results: We found 1218 overviews published from 2000 to 2020; the majority (73%) were published in the most recent 5-year period. We extracted a selection of these overviews (n=541; 44%) dated from 2000 to 2018. The 541 overviews were published in 307 journals; CDSR (8%), PLOS ONE (3%) and Sao Paulo Medical Journal (2%) were the most prevalent. The majority (70%) were published in journals with impact factors between 0.05 and 3.97. We found a mean citation count of 10 overviews per year, published in journals with a mean JIF of 4.4. In multivariable analysis, overviews with a high number of citations and JIFs had more authors, larger sample sizes, were open access and reported the funding source. Conclusions: An 8-fold increase in the number of overviews was found between 2009 and 2020. We identified 332 overviews published in 2020, which is equivalent to 1 overview published per day. Overviews perform above average for the journals in which they publish.

show abstract

Cross-sectional analysis of bibliometrics and altmetrics: comparing the impact of qualitative and quantitative articles in the British Medical Journal

Cited by 18 publications

References 42 publications

Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

Characteristics, quality and volume of the first 5 months of the COVID-19 evidence synthesis infodemic: a meta-research study

Relationship between altmetric and bibliometric indicators across academic social sites in article-level: the case of Iranian researchers' “Top Papers” in clinical medicine

Bibliometric study of ‘overviews of systematic reviews’ of health interventions: evaluation of prevalence, citation and impact factor

Contact Info

Product

Resources

About