General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators

Abstract

The data collection and reporting approaches of four major altmetric data aggregators are studied. The main aim of this study is to understand how differences in social media tracking and data collection methodologies can have effects on the analytical use of altmetric data. For this purpose, discrepancies in the metrics across aggregators have been studied in order to understand how the methodological choices adopted by these aggregators can explain the discrepancies found. Our results show that different forms of accessing the data from diverse social media platforms, together with different approaches of collecting, processing, summarizing, and updating social media metrics cause substantial differences in the data and metrics offered by these aggregators. These results highlight the importance that methodological choices in the tracking, collecting, and reporting of altmetric data can have in the analytical value of the data. Some recommendations for altmetric users and data aggregators are proposed and discussed.

Altmetrics data providers: A meta-analysis review of the coverage of metrics and publication

“…CED also describes a short activity period (2017-2018), but with significant increases. Thus, in 2017, it showed an average coverage of 9.1% (Ortega, 2018a;2018b;Costas, 2018), which rose to 21.2% in 2018 (Ortega, 2019).…”

“…However, from 2016 to 2017 only three services were used, Mendeley, Altmetric.com and PlumX. Because Lagotto is not implemented in important publishers other than PLoS, it became a limited tool for altmetric studies and was used only in one-time cases Costas, 2018). ImpactStory stops provide a public API, making it hard to obtain random representative samples.…”

“…This percentage descended (10.4%) when they repeated the same study one year later . From 2016 to 2019, coverage increases considerably to nearly 100% (Meschede;Siebenlist, 2018;Costas, 2018;Ortega, 2018a;Ortega, 2019;Gorraiz et al, 2019). The only exception is Torres-Salinas et al, (2017a), who obtained a coverage of 77.7% because their sample was limited to books.…”

“…Lagotto again shows a high main percentage (4.5%) because it is only representative of PLoS journals. Lastly, CED was used in 2017 (Ortega, 2018b;Costas, 2018), resulting in an average of 1.7%, similar to PlumX and Altmetric. com in that year.…”

Publons Peer Evaluation Metrics are not Reliable Measures of Quality or Impact

“…Second, in what ways do the data providers used in scientometric analysis provide discrepant data? Zahedi and Costas (2018), for instance, conducted the most exhaustive comparison of altmetric providers and found that coverage of publications between them varies widely.…”