Opioid death projections with AI-based forecasts using social media language

Abstract

Targeting of location-specific aid for the U.S. opioid epidemic is difficult due to our inability to accurately predict changes in opioid mortality across heterogeneous communities. AI-based language analyses, having recently shown promise in cross-sectional (between-community) well-being assessments, may offer a way to more accurately longitudinally predict community-level overdose mortality. Here, we develop and evaluate, TrOP (Transformer for Opiod Prediction), a model for community-specific trend projection that uses community-specific social media language along with past opioid-related mortality data to predict future changes in opioid-related deaths. TOP builds on recent advances in sequence modeling, namely transformer networks, to use changes in yearly language on Twitter and past mortality to project the following year’s mortality rates by county. Trained over five years and evaluated over the next two years TrOP demonstrated state-of-the-art accuracy in predicting future county-specific opioid trends. A model built using linear auto-regression and traditional socioeconomic data gave 7% error (MAPE) or within 2.93 deaths per 100,000 people on average; our proposed architecture was able to forecast yearly death rates with less than half that error: 3% MAPE and within 1.15 per 100,000 people.

PROVIDENT: Development and Validation of a Machine Learning Model to Predict Neighborhood-level Overdose Risk in Rhode Island

“…Data sources not accessible for use during modeling but with established signals for overdose risk, for example, methadone treatment data, 51 may prioritize out-of-treatment populations in model predictions. While emerging research demonstrates the capacity for social media data to inform area-level overdose risk, 52 we restricted our data sources to those widely accessible to public health practitioners.…”

Opportunities of AI-powered applications in anesthesiology to enhance patient safety

“…These large transformer models can adapt to new tasks without or with minimal explicit training 13 . While these types of models can accomplish complex medical knowledge tasks with strong performance, 14 direct implementation in clinical care requires further investigation.…”

Automatic implicit motives codings are as accurate as humans’, cheaper, and 99% faster

“…Researchers who have applied LLM-based predictive models have been able to predict corresponding rating scales from language assessment (rating vs describing well-being) approaching the theoretical upper limit (i.e., r = .85; Kjell et al, 2022), addiction treatment dropout from social media language beyond rating scales (Curtis et al, 2023), and US county-level opioid mortality with less than half the error as compared to models based on socioeconomic data (Matero et al, 2023).…”

“…Interestingly, noted that they experimented with transformers-based language models but that their initial results did not improve compared to their convolutional neural networks. However, we wanted to thoroughly evaluate LLM-based approaches, considering the performance of LLMs in the past years, especially in strongly context-dependent settings (Curtis et al, 2023;Matero et al, 2023;Wang et al, 2019). PSE stories are context-dependent, which is one reason coders undergo rigorous training.…”

“…These consist of several 1,000 latent dimensions (or features) over 10s of layers that capture the meaning of the language, which are referred to as embeddings. Predictive models based on LLM embeddings have shown strong predictive power in psychology settings (e.g., Curtis et al, 2023;Kjell et al, 2022;Matero et al, 2023).…”