Automatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition

Weissenbacher, Davy; O’Connor, Karen; Rawal, Siddharth; Zhang, Yu; Tsai, Richard Tzong-Han; Miller, Timothy A.; Xu, Dongfang; Anderson, Carol; Liu, Bo; Han, Qing; Zhang, Jinfeng; Kulev, Igor; Köprü, Berkay; Rodriguez‐Esteban, Raul; Özkırımlı, Elif; Ayach, Ammer; Roller, Roland; Piccolo, Stephen R.; Han, Peijin; Vydiswaran, V. G. Vinod; Tekumalla, Ramya; Banda, Juan M.; Bagherzadeh, Parsa; Bergler, Sabine; Silva, João Figueira; Almeida, Tiago; Martı́nez, Paloma; Zavala, Renzo Rivera; Wang, Chen-Kai; Dai, Hong-Jie; Hernandez, Luis Alberto Robles; Gonzalez-Hernandez, Graciela

doi:10.1093/database/baac108

Cited by 4 publications

(5 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…and machine learning (ML) methods have facilitated automatic detection of relevant mentions [13,14]. These methods face numerous challenges, such as the highly informal language used on social media, and extracting user-expressed ADE concepts which are usually descriptive and nontechnical [15,16].…”

Section: Introductionmentioning

confidence: 99%

“…These methods face numerous challenges, such as the highly informal language used on social media, and extracting user-expressed ADE concepts which are usually descriptive and nontechnical [15,16]. NLP has played a crucial role in overcoming some of these barriers encountered in identifying ADE mentions [13,14]. While technological methods continue to advance [17][18][19][20][21], the practical utility of social media for identifying adverse events requires further demonstration [22],…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The value of social media analysis for adverse events detection and pharmacovigilance: a Scoping Review (Preprint)

Golder,

O'Connor,

Wang

et al. 2024

JMIR Public Health and Surveillance

Self Cite

View full text Add to dashboard Cite

Background: Adverse drug events pose a significant public health burden leading to hospitalisation, disability, and death. Even those adverse events categorised as non-serious can severely impact on patient quality of life, adherence, and persistence. Monitoring medication safety is challenging. Online patient reports on social media may be a useful supplementary source of realworld data. Despite the growth of sophisticated techniques for identifying adverse events using social media data, a consensus has not been reached as to the value of social media in relation to more traditional data sources.Objective: To evaluate and characterise the utility of social media analysis in adverse drug event detection and pharmacovigilance as compared to other data sources (such as spontaneous reporting systems and the clinical literature). Methods:In this scoping review, we searched 11 bibliographical databases and Google Scholar, followed by handsearching and forward and backward citation searching. Each records was screened by two independent reviewers at both title/abstract stage and the full-text screening stage. Studies were included if they 1) used any type of social media (such as Twitter or patient forums) to detect any adverse event associated with any drug medication and 2) compared the results ascertained from social media to any other data source. Study information was collated using a piloted data extraction sheet . Data were extracted on 1) the adverse events and drugs searched for and included, 2) the methods used (such as machine learning), 3) social media data source, 4) volume of data analysed, 5) limitations of the methodology, 6) availability of data and code, 7) comparison data source and comparison methods, 8) results, including the volume of adverse events, and how adverse events found compared to other data sources in their seriousness, frequencies, and expectedness/novelty (new vs. known knowledge) and 9) conclusions.Results: Of the 6538 unique records screened, 73 publications representing 60 studies and a wide variety of methods met our inclusion criteria. The most common social media platforms used were Twitter and online health forums. The most common comparator data source was spontaneous reporting systems, although other comparisons were made such as with scientific literature and product labels. Although similar patterns of adverse event reporting tended to be identified, the frequencies were lower in social media. Social media data were found useful in identifying new or unexpected adverse events and adverse events in a more timely manner.Conclusions: There is a large body of research comparing adverse events from social media to other sources. Most studies advocated the use of social media as an adjunct to traditional data sources. Some indicated the value of social media in understanding patient perspectives such as the impact of adverse events, which could be better explored.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

The value of social media analysis for adverse events detection and pharmacovigilance: a Scoping Review (Preprint)

Golder,

O'Connor,

Wang

et al. 2024

JMIR Public Health and Surveillance

Self Cite

View full text Add to dashboard Cite

show abstract

“…It was believed that social media data could be used to identify new signals or signals earlier than conventional methods [ 12 ]. To cope with the enormous amounts of text-based information posted on social media, natural language processing (NLP) and machine learning (ML) methods for automatic detection of mentions are continually being developed [ 13 , 14 ]. These methods have to overcome many challenges, for instance, the language in social media is highly informal, and user-expressed concepts are often nontechnical, descriptive, and challenging to extract [ 15 , 16 ].…”

Section: Introductionmentioning

confidence: 99%

“…These methods have to overcome many challenges, for instance, the language in social media is highly informal, and user-expressed concepts are often nontechnical, descriptive, and challenging to extract [ 15 , 16 ]. NLP has been particularly instrumental in overcoming some of the barriers to identify adverse event mentions [ 13 , 14 ]. However, while the technological methods have advanced, the use of social media in identifying adverse events has not been sufficiently demonstrated, and thus, the debate as to whether (and if so, how) social media can enhance pharmacovigilance is still not resolved.…”

Section: Introductionmentioning

confidence: 99%

The Role of Social Media for Identifying Adverse Drug Events Data in Pharmacovigilance: Protocol for a Scoping Review

Golder¹,

O’Connor²,

Wang³

et al. 2023

JMIR Res Protoc

Self Cite

View full text Add to dashboard Cite

Background Adverse drug events (ADEs) are a considerable public health burden resulting in disability, hospitalization, and death. Even those ADEs deemed nonserious can severely impact a patient’s quality of life and adherence to intervention. Monitoring medication safety, however, is challenging. Social media may be a useful adjunct for obtaining real-world data on ADEs. While many studies have been undertaken to detect adverse events on social media, a consensus has not yet been reached as to the value of social media in pharmacovigilance or its role in pharmacovigilance in relation to more traditional data sources. Objective The aim of the study is to evaluate and characterize the use of social media in ADE detection and pharmacovigilance as compared to other data sources. Methods A scoping review will be undertaken. We will search 11 bibliographical databases as well as Google Scholar, hand-searching, and forward and backward citation searching. Records will be screened in Covidence by 2 independent reviewers at both title and abstract stage as well as full text. Studies will be included if they used any type of social media (such as Twitter or patient forums) to detect any type of adverse event associated with any type of medication and then compared the results from social media to any other data source (such as spontaneous reporting systems or clinical literature). Data will be extracted using a data extraction sheet piloted by the authors. Important data on the types of methods used (such as machine learning), any limitations of the methods used, types of adverse events and drugs searched for and included, availability of data and code, details of the comparison data source, and the results and conclusions will be extracted. Results We will present descriptive summary statistics as well as identify any patterns in the types and timing of ADEs detected, including but not limited to the similarities and differences in what is reported, gaps in the evidence, and the methods used to extract ADEs from social media data. We will also summarize how the data from social media compares to conventional data sources. The literature will be organized by the data source for comparison. Where possible, we will analyze the impact of the types of adverse events, the social media platform used, and the methods used. Conclusions This scoping review will provide a valuable summary of a large body of research and important information for pharmacovigilance as well as suggest future directions of further research in this area. Through the comparisons with other data sources, we will be able to conclude the added value of social media in monitoring adverse events of medications, in terms of type of adverse events and timing. International Registered Report Identifier (IRRID) PRR1-10.2196/47068

show abstract

“…Our contributions are: 1) the release of 212 Twitter timelines annotated with medication spans for benchmarking medications extraction in Twitter, to be done as a BioCreative shared-task [18]; 2) the design of a fast and efficient classifier to detect tweets mentioning medications; 3) an intrinsic evaluation of the classifier as well an extrinsic evaluation when it is used to help the extraction of the medications' spans.…”

Section: Introductionmentioning

confidence: 99%

Addressing Extreme Imbalance for Detecting Medications Mentioned in Twitter User Timelines

Weissenbacher

Rawal

Magge

et al. 2021

Artificial Intelligence in Medicine

View full text Add to dashboard Cite

Tweets mentioning medications are valuable for efforts in digital epidemiology to supplement traditional methods of monitoring public health. A major obstacle, however, is to differentiate them from the large majority of tweets on other topics posted in a user's timeline: solving the infamous 'needle in a haystack' problem. While deep learning models have significantly improved classification, their performance and inference processing time remain low on extremely imbalanced corpora where the tweets of interest are less than 1% of all tweets. In this study, we empirically evaluate under-sampling, fine-tuning, and filtering heuristics to train such classifiers. Using a corpus of 212 Twitter timelines (181,607 tweets with only 0.2% tweets mentioning a medication), our results show that combining these heuristics is necessary to impact the classifier's performance. In our intrinsic evaluation, a classifier based on a lexicon and a BERT-base neural network achieved a 0.838 F1-score, a score similar to the ones of the best existing classifier, but it processed the corpus 28 times faster -a positive result, since processing speed is still a roadblock to deploying classifiers on large cohorts of Twitter users needed for pharmacovigilance. In our extrinsic evaluation, our classifier helped a labeler to extract the spans of medications more accurately and achieved a 0.76 Strict F1-score. To the best of our knowledge, this is the first evaluation of medications extraction in Twitter timelines and it establishes the first benchmark for future studies.

show abstract

Automatic Extraction of Medication Mentions from Tweets—Overview of the BioCreative VII Shared Task 3 Competition

Cited by 4 publications

References 34 publications

The value of social media analysis for adverse events detection and pharmacovigilance: a Scoping Review (Preprint)

The value of social media analysis for adverse events detection and pharmacovigilance: a Scoping Review (Preprint)

The Role of Social Media for Identifying Adverse Drug Events Data in Pharmacovigilance: Protocol for a Scoping Review

Addressing Extreme Imbalance for Detecting Medications Mentioned in Twitter User Timelines

Contact Info

Product

Resources

About