Biases in using social media data for public health surveillance: A scoping review

Zhao, Yunpeng; He, Xiaowen; Feng, Zhenghe; Bost, Sarah; Prosperi, Mattia; Wu, Yonghui; Guo, Yi; Bian, Jiang

doi:10.1016/j.ijmedinf.2022.104804

Cited by 15 publications

(8 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Given the anonymity of Reddit, it could be a valuable source to survey questions about sensitive topics 6 . Additional social media sites could be included in future studies to expand the participant base 15 . As new substances emerge in the drug market, Reddit could be a resource to study these substances with the goal of informing public health.…”

Section: Discussionmentioning

confidence: 99%

Self-reported Xylazine Experiences: A Mixed-methods Study of Reddit Subscribers

et al. 2023

View full text Add to dashboard Cite

Objectives Xylazine is an α2-agonist increasingly prevalent in the illicit drug supply. Our objectives were to curate information about xylazine through social media from people who use drugs (PWUDs). Specifically, we sought to answer the following: (1) What are the demographics of Reddit subscribers reporting exposure to xylazine? (2) Is xylazine a desired additive? And (3) what adverse effects of xylazine are PWUDs experiencing? Methods Natural language processing (NLP) was used to identify mentions of “xylazine” from posts by Reddit subscribers who also posted on drug-related subreddits. Posts were qualitatively evaluated for xylazine-related themes. A survey was developed to gather additional information about the Reddit subscribers. This survey was posted on subreddits that were identified by NLP to contain xylazine-related discussions from March 2022 to October 2022. Results Seventy-six posts were extracted via NLP from 765,616 posts by 16,131 Reddit subscribers (January 2018 to August 2021). People on Reddit described xylazine as an unwanted adulterant in their opioid supply. Sixty-one participants completed the survey. Of those who disclosed their location, 25 of 50 participants (50%) reported locations in the Northeastern United States. The most common route of xylazine use was intranasal use (57%). Thirty-one of 59 (53%) reported experiencing xylazine withdrawal. Frequent adverse events reported were prolonged sedation (81%) and increased skin wounds (43%). Conclusions Among respondents on these Reddit forums, xylazine seems to be an unwanted adulterant. People who use drugs may be experiencing adverse effects such as prolonged sedation and xylazine withdrawal. This seemed to be more common in the Northeast.

show abstract

Section: Discussionmentioning

confidence: 99%

Self-reported Xylazine Experiences: A Mixed-methods Study of Reddit Subscribers

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Social media data has been acknowledged to exhibit demographic or geographic biases, which can impact the model’s generalizability ( 48 ). It’s important to note the potential bias inherent in both the human annotated ADE-corpus-V2 (training dataset) and the SMM4H social media tweets (evaluation dataset).…”

Section: Discussionmentioning

confidence: 99%

BERT-based language model for accurate drug adverse event extraction from social media: implementation, evaluation, and contributions to pharmacovigilance practices

Dong,

Guo,

Liu

et al. 2024

Front. Public Health

View full text Add to dashboard Cite

IntroductionSocial media platforms serve as a valuable resource for users to share health-related information, aiding in the monitoring of adverse events linked to medications and treatments in drug safety surveillance. However, extracting drug-related adverse events accurately and efficiently from social media poses challenges in both natural language processing research and the pharmacovigilance domain.MethodRecognizing the lack of detailed implementation and evaluation of Bidirectional Encoder Representations from Transformers (BERT)-based models for drug adverse event extraction on social media, we developed a BERT-based language model tailored to identifying drug adverse events in this context. Our model utilized publicly available labeled adverse event data from the ADE-Corpus-V2. Constructing the BERT-based model involved optimizing key hyperparameters, such as the number of training epochs, batch size, and learning rate. Through ten hold-out evaluations on ADE-Corpus-V2 data and external social media datasets, our model consistently demonstrated high accuracy in drug adverse event detection.ResultThe hold-out evaluations resulted in average F1 scores of 0.8575, 0.9049, and 0.9813 for detecting words of adverse events, words in adverse events, and words not in adverse events, respectively. External validation using human-labeled adverse event tweets data from SMM4H further substantiated the effectiveness of our model, yielding F1 scores 0.8127, 0.8068, and 0.9790 for detecting words of adverse events, words in adverse events, and words not in adverse events, respectively.DiscussionThis study not only showcases the effectiveness of BERT-based language models in accurately identifying drug-related adverse events in the dynamic landscape of social media data, but also addresses the need for the implementation of a comprehensive study design and evaluation. By doing so, we contribute to the advancement of pharmacovigilance practices and methodologies in the context of emerging information sources like social media.

show abstract

“…While it is not possible for us to influence the demographic distribution of Twitter, in the future we will use automatic cohort characterization strategies for estimating the distributions of race, gender identity, and age-group of the cohort [ 46 ]. A careful study of the biases present in our social media cohort data will also be important future work [ 47 ]. Importantly, future studies relying on or utilizing our outputs/findings should be mindful of the many and evolving biases in social media data and the potential harms of propagating inequity.…”

Section: Discussionmentioning

confidence: 99%

#ChronicPain: Automated Building of a Chronic Pain Cohort from Twitter Using Machine Learning

et al. 2023

View full text Add to dashboard Cite

Background: Due to the high burden of chronic pain, and the detrimental public health consequences of its treatment with opioids, there is a high-priority need to identify effective alternative therapies. Social media is a potentially valuable resource for knowledge about self-reported therapies by chronic pain sufferers. Methods: We attempted to (a) verify the presence of large-scale chronic pain-related chatter on Twitter, (b) develop natural language processing and machine learning methods for automatically detecting self-disclosures, (c) collect longitudinal data posted by them, and (d) semiautomatically analyze the types of chronic pain-related information reported by them. We collected data using chronic pain-related hashtags and keywords and manually annotated 4,998 posts to indicate if they were self-reports of chronic pain experiences. We trained and evaluated several state-of-the-art supervised text classification models and deployed the best-performing classifier. We collected all publicly available posts from detected cohort members and conducted manual and natural language processing-driven descriptive analyses. Results: Interannotator agreement for the binary annotation was 0.82 (Cohen’s kappa). The RoBERTa model performed best (F 1 score: 0.84; 95% confidence interval: 0.80 to 0.89), and we used this model to classify all collected unlabeled posts. We discovered 22,795 self-reported chronic pain sufferers and collected over 3 million of their past posts. Further analyses revealed information about, but not limited to, alternative treatments, patient sentiments about treatments, side effects, and self-management strategies. Conclusion: Our social media based approach will result in an automatically growing large cohort over time, and the data can be leveraged to identify effective opioid-alternative therapies for diverse chronic pain types.

show abstract

Biases in using social media data for public health surveillance: A scoping review

Cited by 15 publications

References 41 publications

Self-reported Xylazine Experiences: A Mixed-methods Study of Reddit Subscribers

Self-reported Xylazine Experiences: A Mixed-methods Study of Reddit Subscribers

BERT-based language model for accurate drug adverse event extraction from social media: implementation, evaluation, and contributions to pharmacovigilance practices

#ChronicPain: Automated Building of a Chronic Pain Cohort from Twitter Using Machine Learning

Contact Info

Product

Resources

About