Social media mining for birth defects research: A rule-based, bootstrapping approach to collecting data for rare health-related events on Twitter

Abstract

Background: Although birth defects are the leading cause of infant mortality in the United States, methods for observing human pregnancies with birth defect outcomes are limited. Objective: The primary objectives of this study were (i) to assess whether rare health-related events—in this case, birth defects—are reported on social media, (ii) to design and deploy a natural language processing (NLP) approach for collecting such sparse data from social media, and (iii) to utilize the collected data to discover a cohort of women whose pregnancies with birth defect outcomes could be observed on social media for epidemiological analysis. Methods: To assess whether birth defects are mentioned on social media, we mined 432 million tweets posted by 112,647 users who were automatically detected via their public announcements of pregnancies on Twitter. To retrieve tweets that mention birth defects, we developed a rule-based, bootstrapping approach, which relies on a lexicon, lexical variants generated from the lexicon entries, regular expressions, post-processing, and manual analysis guided by distributional properties. To identify users whose pregnancies with birth defect outcomes could be observed for epidemiological analysis, inclusion criteria were (i) tweets indicating that the user’s child has a birth defect, and (ii) accessibility to the user’s tweets during pregnancy. We conducted a semi-automatic evaluation to estimate the recall of the tweet-collection approach, and performed a preliminary assessment of the prevalence of selected birth defects among the pregnancy cohort derived from Twitter. Results: We manually annotated 16,822 retrieved tweets, distinguishing tweets indicating that the user’s child has a birth defect (true positives) from tweets that merely mention birth defects (false positives). Inter-annotator agreement was substantial: κ = 0.79 (Cohen’s kappa). Analyzing the timelines of the 646 users whose tweets were true positives resulted in the discovery of 195 users that met the inclusion criteria. Congenital heart defects are the most common type of birth defect reported on Twitter, consistent with findings in the general population. Based on an evaluation of 4,169 tweets retrieved using alternative text mining methods, the recall of the tweet-collection approach was 0.95. Conclusions: Our contributions include (i) evidence that rare health-related events are indeed reported on Twitter, (ii) a generalizable, systematic NLP approach for collecting sparse tweets, (iii) a semi-automatic method to identify undetected tweets (false negatives), and (iv) a collection of publicly available tweets by pregnant users with birth defect outcomes, which could be used for future epidemiological analysis. In future work, the annotated tweets could be used to train machine learning algorithms to automatically identify users reporting birth defect outcomes, enabling the large-scale use of social media mining as a complementary method for such epidemiological research.

Pregnancy and health in the age of the Internet: A content analysis of online “birth club” forums

“…An alternate approach to studying how women utilize the Internet during pregnancy is to examine the online content that women generate. Our prior work focused on gathering pregnancy-related information from a generic social media platform such as Twitter, whereby a cohort of pregnant women was identified via their self-reports of pregnancy and their timelines (all publicly available tweets) during pregnancy were analyzed in a case-control study of birth defects [25][26][27]. Other researchers have identified pregnant women based on their search queries, and determined their time-dependent search queries throughout pregnancy [28].…”

An annotated data set for identifying women reporting adverse pregnancy outcomes on Twitter

“…We handcrafted 11 regular expressions to retrieve tweets that mention adverse pregnancy outcomes, from a database containing more than 400 million public tweets posted by more than 100,000 users who have announced their pregnancy on Twitter [7] . These query patterns were designed to account for the various ways adverse pregnancy outcomes may be linguistically expressed on social media—for example, reporting a miscarriage or stillbirth through the use of rainbow baby (Pattern 2) or hashtags such as #babyloss, #pregnancyloss, #iam1in4 , or #waveoflight (Pattern 9), learned through an iterative process of manually reviewing tweets matched by other query patterns [8] . Similarly, preterm birth, for example, may be expressed by the user referring to her baby as a preemie (Pattern 4), or by reporting that her baby was born at less than 37 weeks of gestation (Pattern 5) or more than three weeks early (Pattern 7).…”

“…Initially, the query patterns focused on the recall of “outcome” tweets. Given the relatively low prevalence of adverse pregnancy outcomes in the general population, and our related work [8] suggesting that they may be under-reported on Twitter, high-precision query patterns would result in a sparse representation of “outcome” tweets; however, many of the preliminary regular expressions would have led to a high degree of class-imbalanced data [12] for training machine learning algorithms to automatically detect “outcome” tweets. Thus, to balance recall and precision, the final regular expressions required a reference to the user (e.g., I, our ), a child (e.g., daughter, baby ), or birth (e.g., born, welcome ) preceding the mention of an adverse pregnancy outcome, while allowing any number of characters to occur between.…”

“…These “outcome” tweets can be used to explore patient experiences and perceptions of adverse pregnancy outcomes, and can direct researchers to the users’ broader timelines—tweets posted by a user over time—for observational studies. Our past work demonstrates the analysis of timelines for selecting a study population [8] and conducting a case-control study [9] of users reporting that their child has a birth defect. For larger-scale studies, the full annotated corpus can be used to train supervised machine learning algorithms to automatically identify additional users reporting adverse pregnancy outcomes on Twitter.…”

A proof-of-concept study of extracting patient histories for rare/intractable diseases from social media

“…Consequently, these disease communities use social media platforms to try to find other patients with similar health problems or expertise about their rare condition, sharing manifold types of information-including symptoms, treatments, side effects, and other diseases and activities-that go beyond what is normally captured in a clinical setting or patient registry [9]. Recently, Klein et al [10] mined Twitter to collect data on rare health-related events reported by patients, and showed that this social media platform was useful for gathering patient-centric information that could be used for future epidemiological analyses. Our hypothesis was that data from RD patient histories posted on social media would capture patients' perspectives of their health status, which may be valuable for research into ways of helping undiagnosed patients by accelerating the timeline to diagnosis and treatment.…”