Identifying Cases of Shoulder Injury Related to Vaccine Administration (SIRVA) in the United States: Development and Validation of a Natural Language Processing Method

Abstract

Background Shoulder injury related to vaccine administration (SIRVA) accounts for more than half of all claims received by the National Vaccine Injury Compensation Program. However, due to the difficulty of finding SIRVA cases in large health care databases, population-based studies are scarce. Objective The goal of the research was to develop a natural language processing (NLP) method to identify SIRVA cases from clinical notes. Methods We conducted the study among members of a large integrated health care organization who were vaccinated between April 1, 2016, and December 31, 2017, and had subsequent diagnosis codes indicative of shoulder injury. Based on a training data set with a chart review reference standard of 164 cases, we developed an NLP algorithm to extract shoulder disorder information, including prior vaccination, anatomic location, temporality and causality. The algorithm identified 3 groups of positive SIRVA cases (definite, probable, and possible) based on the strength of evidence. We compared NLP results to a chart review reference standard of 100 vaccinated cases. We then applied the final automated NLP algorithm to a broader cohort of vaccinated persons with a shoulder injury diagnosis code and performed manual chart confirmation on a random sample of NLP-identified definite cases and all NLP-identified probable and possible cases. Results In the validation sample, the NLP algorithm had 100% accuracy for identifying 4 SIRVA cases and 96 cases without SIRVA. In the broader cohort of 53,585 vaccinations, the NLP algorithm identified 291 definite, 124 probable, and 52 possible SIRVA cases. The chart-confirmation rates for these groups were 95.5% (278/291), 67.7% (84/124), and 17.3% (9/52), respectively. Conclusions The algorithm performed with high sensitivity and reasonable specificity in identifying positive SIRVA cases. The NLP algorithm can potentially be used in future population-based studies to identify this rare adverse event, avoiding labor-intensive chart review validation.

Architecture Assessment of the Chilean Epidemiological Surveillance System for Notifiable Diseases (EPIVIGILA): Qualitative Study

“…Thus, from the beginning, the EPIVIGILA platform was expected to show high accuracy, process a large number of daily notifications, and generate a robust database, all while facing a pandemic phenomenon that impacted different countries at the same time. This sanitary scenario witnessed a worldwide deployment of new information technologies, which proved to be efficient helping tools for public health control [11][12][13].…”

Artificial intelligence in healthcare: Opportunities come with landmines

“… 1 2 The seamless integration of AI technologies in radiology is reshaping diagnostic precision while NLP’s capacity to decipher and enhance safety protocols heralds a new era in healthcare innovation. 3 4 …”