Quantifying and contextualizing the impact of bioRxiv preprints through automated social media audience segmentation

Abstract

Engagement with scientific manuscripts is frequently facilitated by Twitter and other social media platforms. As such, the demographics of a paper's social media audience provide a wealth of information about how scholarly research is transmitted, consumed, and interpreted by online communities. By paying attention to public perceptions of their publications, scientists can learn whether their research is stimulating positive scholarly and public thought. They can also become aware of potentially negative patterns of interest from groups that misinterpret their work in harmful ways, either willfully or unintentionally, and devise strategies for altering their messaging to mitigate these impacts. In this study, we collected 331,696 Twitter posts referencing 1,800 highly tweeted bioRxiv preprints and leveraged topic modeling to infer the characteristics of various communities engaging with each preprint on Twitter. We agnostically learned the characteristics of these audience sectors from keywords each user's followers provide in their Twitter biographies. We estimate that 96% of the preprints analyzed are dominated by academic audiences on Twitter, suggesting that social media attention does not always correspond to greater public exposure. We further demonstrate how our audience segmentation method can quantify the level of interest from nonspecialist audience sectors such as mental health advocates, dog lovers, video game developers, vegans, bitcoin investors, conspiracy theorists, journalists, religious groups, and political constituencies. Surprisingly, we also found that 10% of the preprints analyzed have sizable (>5%) audience sectors that are associated with right-wing white nationalist communities. Although none of these preprints appear to intentionally espouse any right-wing extremist messages, cases exist in which extremist appropriation comprises more than 50% of the tweets referencing a given preprint. These results present unique opportunities for improving and contextualizing the public discourse surrounding scientific research.

Preprinting the COVID-19 pandemic

“…Indeed, even before this warning message was posted, preprints were receiving significant amounts of attention. Twitter has been a particularly notable outlet for communication of preprints, a finding echoed by a recent study on the spread of the wider (i.e., not limited to preprints) COVID-19 research field on Twitter, which found that COVID-19 research was being widely disseminated and driven largely by academic Twitter users [31,32]. Nonetheless, the relatively weak correlation found between citations and other indicators of online sharing (Fig 6H) suggests that the interests of scientists versus the broader public differ significantly: of the articles in the top 10 most shared on twitter, in news articles or on blogs, only one is ranked amongst the top 10 most cited articles (Supplemental Fig.…”

“…Hashtags associated with individual, highly tweeted preprints reveal some emergent themes that suggest communication of certain preprints can also extend well beyond scientific audiences (Supplemental Fig. 4A) [32]. These range from good public health practice (“#washyourhands”) to right-wing philosophies (#chinalies), conspiracy theories (“#fakenews” and “#endthelockdown”) and xenophobia (“#chinazi”).…”

The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape

“…Twitter has been a particularly notable outlet for communication of preprints, a finding echoed by a recent study on the spread of the wider (i.e., not limited to preprints) COVID-19 research field on Twitter, which found that COVID-19 research was being widely disseminated and driven largely by academic Twitter users [33,34]. Nonetheless, the relatively weak correlation found between citations and other indicators of online sharing (Fig 6H ) suggests that the interests of scientists versus the broader public differ significantly: Of the articles in the top 10 most shared on Twitter, in news articles or on blogs, only one is ranked among the top 10 most cited articles (S4B Fig) . Hashtags associated with individual, highly tweeted preprints reveal some emergent themes that suggest communication of certain preprints can also extend well beyond scientific audiences (S4A Fig) [34]. These range from good public health practice ("#washyourhands") to right-wing philosophies (#chinalies), conspiracy theories ("#fakenews" and "#endthelockdown"), and xenophobia ("#chinazi").…”

Preprints: Their Evolving Role in Science Communication

“…Media coverage can be extremely influential, but it is not the only way preprints reach a wide audience. Preprints can be broadly disseminated on social media; for example, a study quantified the level of interest garnered by several preprint and showed they attracted reactions from a diverse range of non-specialist audience sectors such as mental health advocates, dog lovers, video game developers, vegans, bitcoin investors, conspiracy theorists, journalists, religious groups, and political constituencies (Carlson & Harris, 2020) . While this broad range of audiences, and the potential risk of misappropriation by certain groups is likely not unique to preprints, the authors of the study highlight the need for researchers to be mindful not only of how they communicate, but also how they design their studies.…”