Do Altmetrics Work? Twitter and Ten Other Social Web Services

Abstract

Altmetric measurements derived from the social web are increasingly advocated and used as early indicators of article impact and usefulness. Nevertheless, there is a lack of systematic scientific evidence that altmetrics are valid proxies of either impact or utility although a few case studies have reported medium correlations between specific altmetrics and citation rates for individual journals or fields. To fill this gap, this study compares 11 altmetrics with Web of Science citations for 76 to 208,739 PubMed articles with at least one altmetric mention in each case and up to 1,891 journals per metric. It also introduces a simple sign test to overcome biases caused by different citation and usage windows. Statistically significant associations were found between higher metric scores and higher citations for articles with positive altmetric scores in all cases with sufficient evidence (Twitter, Facebook wall posts, research highlights, blogs, mainstream media and forums) except perhaps for Google+ posts. Evidence was insufficient for LinkedIn, Pinterest, question and answer sites, and Reddit, and no conclusions should be drawn about articles with zero altmetric scores or the strength of any correlation between altmetrics and citations. Nevertheless, comparisons between citations and metric values for articles published at different times, even within the same year, can remove or reverse this association and so publishers and scientometricians should consider the effect of time when using altmetrics to rank articles. Finally, the coverage of all the altmetrics except for Twitter seems to be low and so it is not clear if they are prevalent enough to be useful in practice.

Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research

“…Twitter seems to have a social orientation (Costas et al 2015b) and more frequency cites scholarly publications than do other altmetric sources, including academic blogs, Facebook and Google+ (Costas et al 2015a;Haustein et al 2015;Thelwall et al 2013). There is also evidence that a considerable number of Twitter mentions occur shortly after research is published (Eysenbach 2011; Priem and Costello 2010) and they seem to reflect attention, popularity or visibility rather than impact, so that Twitter mentions may be an early indicator of the level of attention (including publicity) that articles attract (Haustein et al 2014).…”

The Altmetric Score: A New Measure for Article-Level Dissemination and Impact

“…As a result, highly disseminated articles may be identified within days of their publication. [18][19][20][21] Although altmetrics have certain advantages over citation-based metrics such as the journal impact factor and h-index, the type of impact that they measure, although related, is not the same. We propose that altmetrics be thought of as measures of "disseminative impact," whereas traditional citation-based metrics be considered measures of "scholarly impact."…”

“…Although it is logical that journal impact factor and Altmetric scores correlate with each other, [18][19][20][21] broadly disseminated articles may not be cited frequently, and vice versa. 23,24 As a measure of dissemination, altmetrics may reflect readership among a broader audience of practicing clinicians and not simply impact within academic spheres.…”

“…However, altmetrics' ability to measure disseminative impact quickly and the correlation with citations [18][19][20][21] suggest that altmetrics may serve as a useful complement to journal impact factor.…”

Truebounded, Overbounded, or Underbounded? Scientists’ Personal Publication Lists versus Lists Generated through Bibliographic Information Services

“…However, visibility may not only refer to publication and citation counts within established academic databases. Social media services (like Twitter or Mendeley) can also be used to study the visibility of an author [15], so the use of social media for scientific purposes can increase an author's visibility. There is one further different approach.…”