Do altmetric scores reflect article quality? Evidence from the <scp>UK</scp> Research Excellence Framework 2021

Thelwall, Mike; Kousha, Kayvan; Abdoli, Mahshid; Stuart, David; Makita, Meiko; Wilson, Paul; Levitt, Jonathan M.

doi:10.1002/asi.24751

Cited by 8 publications

(2 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In theory, citations reflect academic impact and Altmetric scores reflect societal impact. However, Altmetric scores have also been shown to be associated with article quality-though less strongly than citation counts (Thelwall et al, 2023)-and, given social media mentions may well be from other researchers or scholarly bodies, the boundaries between academic and societal impact are not clear-cut. Institutions and researchers, through schemes such as the 2013 San Francisco Declaration on Research Assessment, are being encouraged to consider value beyond traditional article and journal bibliometrics when assessing research and researchers.…”

Section: Discussionmentioning

confidence: 99%

Exploring the relationship between traditional bibliometrics and Altmetric scores in the primary care literature

Burrell,

Butler,

Ukoumunne

et al. 2023

Learned Publishing

View full text Add to dashboard Cite

There is some evidence that Altmetric scores correlate with citations in medical research, but this is not consistent across different specialties. No previous studies have examined the association between Altmetric score and citations amongst primary care research journals. The aim of this study was therefore to describe this association. We identified the ten most frequently cited articles published in the top 15 highest impact factor primary care research journals. Article and journal metrics were extracted and summarized using descriptive statistics. We used Spearman's correlation coefficient (rs) and log–log linear regression modelling to examine the relationship between citations and Altmetric score. 150 articles were included with a median of 36.5 (IQR 20–59; range 5–811) citations. We found a positive association between citations and Altmetric score (rs = 0.519; p < 0.001). A unit increase in log Altmetric score was associated with increased log citations [0.175 (95% CI 0.091–0.259, p < 0.001)] in an adjusted linear regression model. The regression findings indicate that increasing Altmetric score by 10% was associated with a 1.68% increase in citation rate. This has implications for how authors, academic institutions and primary care research journals approach dissemination of articles.

show abstract

Section: Discussionmentioning

confidence: 99%

Exploring the relationship between traditional bibliometrics and Altmetric scores in the primary care literature

Burrell,

Butler,

Ukoumunne

et al. 2023

Learned Publishing

View full text Add to dashboard Cite

show abstract

“…Previous studies have developed indicators based on usage data from social network T A B L E 5 The effects of self-archiving preprint on citations in 0-3 years after publication. platforms like Mendeley, Twitter, and others (Dardas et al, 2023;Glanzel & Gorraiz, 2015;Thelwall et al, 2023;Zahedi et al, 2017). Altmetrics indicators are particularly suitable for this study as they quantify the extent to which readers take advantage of the free access provided by self-archiving platforms.…”

Section: Free Access Via Social Networkmentioning

confidence: 99%

The enhanced research impact of self‐archiving platforms: Evidence from bioRxiv

Liu,

Hu,

2024

Asso for Info Science & Tech

View full text Add to dashboard Cite

The proliferation of online self‐archiving platforms has greatly facilitated the dissemination of new research findings. However, the extent to which these platforms have increased the impact of research remains uncertain, especially considering the potential bias of researchers favoring the deposition of higher‐quality preprints. To determine the causal relationship between self‐archiving platforms and research impact, we conducted an investigation using a sample of 5423 published articles from 2018 that had preprints deposited on bioRxiv. These articles were compared to a control group of 7862 similar articles without preprints, identified through text‐mining algorithms. By modeling the differences in forward citations over time between the two groups, we established a positive causal effect of self‐archiving platforms on research impact. However, the effect observed in this study was smaller than that reported in previous studies (18.7% vs. 36%), indicating a more accurate and unbiased result. Additionally, the study explored the underlying mechanisms that contribute to the enhancement of research impact, including early dissemination and increased exposure through social networks facilitated by self‐archiving platforms. Overall, this study expands our understanding of how research is disseminated in the era of Internet platforms.

show abstract

Assessing the influence of research quality on policy citations: Quantitative analysis finds non‐academic factors more likely to influence how papers get cited in SDG policy

Mahfouz,

Capra,

Mulgan

2024

Sustainable Development

View full text Add to dashboard Cite

Addressing the United Nations Sustainable Development Goals (SDGs) requires policymakers to utilise high‐quality scholarly research effectively. This study explores the extent to which bibliometric indicators of research quality—field weighted citation index, journal CiteScore, author h‐index and mentions—influence the likelihood of SDG‐related research being cited in policy documents. Analysing metadata from over 16 million SDG‐related articles revealed modest relationships between selected bibliometric indicators and policy citations. This suggests that bibliometric measures of research quality may not adequately explain policy influence, or that these metrics might not capture the essence of research excellence from a policy perspective. Importantly, the study identifies variations in research utilisation across different SDGs and highlights the potential role of non‐academic factors in determining policy impact. Results provide a framework for understanding the extent to which policymakers are leveraging the best available evidence in their efforts to meet the SDGs.

show abstract

Do altmetric scores reflect article quality? Evidence from the UK Research Excellence Framework 2021

Cited by 8 publications

References 35 publications

Exploring the relationship between traditional bibliometrics and Altmetric scores in the primary care literature

Exploring the relationship between traditional bibliometrics and Altmetric scores in the primary care literature

The enhanced research impact of self‐archiving platforms: Evidence from bioRxiv

Assessing the influence of research quality on policy citations: Quantitative analysis finds non‐academic factors more likely to influence how papers get cited in SDG policy

Contact Info

Product

Resources

About