EventEpi—A natural language processing framework for event-based surveillance

Abstract

According to the World Health Organization (WHO), around 60% of all outbreaks are detected using informal sources. In many public health institutes, including the WHO and the Robert Koch Institute (RKI), dedicated groups of public health agents sift through numerous articles and newsletters to detect relevant events. This media screening is one important part of event-based surveillance (EBS). Reading the articles, discussing their relevance, and putting key information into a database is a time-consuming process. To support EBS, but also to gain insights into what makes an article and the event it describes relevant, we developed a natural language processing framework for automated information extraction and relevance scoring. First, we scraped relevant sources for EBS as done at the RKI (WHO Disease Outbreak News and ProMED) and automatically extracted the articles’ key data: disease, country, date, and confirmed-case count. For this, we performed named entity recognition in two steps: EpiTator, an open-source epidemiological annotation tool, suggested many different possibilities for each. We extracted the key country and disease using a heuristic with good results. We trained a naive Bayes classifier to find the key date and confirmed-case count, using the RKI’s EBS database as labels which performed modestly. Then, for relevance scoring, we defined two classes to which any article might belong: The article is relevant if it is in the EBS database and irrelevant otherwise. We compared the performance of different classifiers, using bag-of-words, document and word embeddings. The best classifier, a logistic regression, achieved a sensitivity of 0.82 and an index balanced accuracy of 0.61. Finally, we integrated these functionalities into a web application called EventEpi where relevant sources are automatically analyzed and put into a database. The user can also provide any URL or text, that will be analyzed in the same way and added to the database. Each of these steps could be improved, in particular with larger labeled datasets and fine-tuning of the learning algorithms. The overall framework, however, works already well and can be used in production, promising improvements in EBS. The source code and data are publicly available under open licenses.

Using digital surveillance tools for near real-time mapping of the risk of infectious disease spread

“…Although the computational time needed for data-preprocessing was negligible (in the order of a few seconds to a minute), extraction of data from a ProMED report can take from 3 to 10 min, depending on the complexity of the report. Ongoing efforts to automate the extraction of quantitative information from ProMED and HealthMap 57 will further enhance the utility of these tools for real-time outbreak analysis. Another factor that could improve the usability of our model in near real time is to reduce the running time of the fitting and forward simulation.…”

Machine and cognitive intelligence for human health: systematic review

“…In terms of monitoring, there are scholars focusing on AI for health monitoring and surveillance in Web-based applications. For example, to support event-based surveillance and understand factors that make an article relevant, Abbood et al [ 77 ] “extracted expert labels from a public health unit that screens online resources every day to train various machine learning models and perform key information extraction as well as relevance scoring on epidemiological texts (p. 1)”. In a Web application integrated with machine learning algorithms designed to monitor pregnancy [ 78 ], users accessed “calculators of baby percentile, period tracker, pregnancy calendar, and baby vaccination schedule (p. 1)”.…”

Year 2020 (with COVID): Observation of Scientific Literature on Clinical Natural Language Processing

“…Indeed, there is a significant number of existing corpora, datasets and resources available in English. Yet, we observe an increasing number of publications dedicated to other languages and a greater variety of languages: Arabic [ 20 ], Chinese [ 21 22 23 24 25 26 ], Croatian [ 27 ], Finnish [ 28 , 29 ], French [ 30 , 31 ], German [ 32 33 34 ], Hebrew [ 35 ], Italian [ 36 37 38 ], Japanese [ 39 , 40 ], Korean [ 41 , 42 ], Norwegian [ 43 ], Portuguese [ 44 ], Spanish [ 45 46 47 48 ], Swedish [ 49 ], and Turkish [ 28 ]. Overall, we believe that the trend observed in previous years is continuing.…”

“…The researches also investigate social networks, focusing on analyzing public opinion and emotions on the COVID pandemics in Twitter posts [ 56 , 57 ], monitoring illicit sales of COVID medication on Twitter and Instagram [ 58 ], and observing COVID symptoms and disease histories collected from a large population in Reddit, which may provide more reliable insights [ 59 ]. Another important issue is that, in the current situation, the surveillance of emerging epidemiological events becomes again very important, as around 60% of all outbreaks are detected using informal sources, which motivated online epidemiological surveillance [ 32 ].…”

“…As noticed above, the social networks continue to provide important information on several issues: public opinion and emotions on the COVID pandemics in Twitter posts [ 56 , 57 ], surveillance of illicit sales of COVID medication on Twitter and Instagram [ 58 ], observation of COVID symptoms and disease histories collected from a large population in Reddit [ 59 ], surveillance of emerging epidemiological events [ 32 ], analysis of HIV-related tweets and of their relation to the HIV incidence [ 77 ], analysis of drug use on Twitter [ 78 ], analysis of developmental crisis episodes during early adulthood in social media [ 75 ].…”