Predicting overdose among individuals prescribed opioids using routinely collected healthcare utilization data

Abstract

Introduction With increasing rates of opioid overdoses in the US, a surveillance tool to identify high-risk patients may help facilitate early intervention. Objective To develop an algorithm to predict overdose using routinely-collected healthcare databases. Methods Within a US commercial claims database (2011-2015), patients with �1 opioid prescription were identified. Patients were randomly allocated into the training (50%), validation (25%), or test set (25%). For each month of follow-up, pooled logistic regression was used to predict the odds of incident overdose in the next month based on patient history from the preceding 3-6 months (time-updated), using elastic net for variable selection. As secondary analyses, we explored whether using simpler models (few predictors, baseline only) or different analytic methods (random forest, traditional regression) influenced performance. Results We identified 5,293,880 individuals prescribed opioids; 2,682 patients (0.05%) had an overdose during follow-up (mean: 17.1 months). On average, patients who overdosed were younger and had more diagnoses and prescriptions. The elastic net model achieved good performance (c-statistic 0.887, 95% CI 0.872-0.902; sensitivity 80.2, specificity 80.1, PPV 0.21, NPV 99.9 at optimal cutpoint). It outperformed simpler models based on few predictors (c-statistic 0.825, 95% CI 0.808-0.843) and baseline predictors only (c-statistic 0.806, 95% CI 0.787-0.26). Different analytic techniques did not substantially influence performance. In

Outcome class imbalance and rare events: An underappreciated complication for overdose risk prediction modeling

“…We first reviewed outcome prevalence in 20 recent papers that developed risk prediction models for opioid-related harms (e.g. fatal opioid overdose, Table 1) [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23]33]. In these papers, non-cases outnumbered cases by a factor of at least 100 unless a deliberate strategy was used to over-sample outcome events; of the 20 papers we chose, four 1).…”

“…To inform design of the simulated data sets, we first reviewed outcome event rates in a convenience sample of published papers predicting opioid‐related adverse effects. We selected 20 recent papers reporting on risk prediction models across a range of opioid‐related outcomes and using a variety of data sources (Table 1) [5–23, 33]. Because our initial search strategy yielded relatively few papers, we selected papers reviewed in a Tseregounis et al .…”

“…To inform design of the simulated data sets, we first reviewed outcome event rates in a convenience sample of published papers predicting opioid-related adverse effects. We selected 20 recent papers reporting on risk prediction models across a range of opioid-related outcomes and using a variety of data sources (Table 1) [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23]33].…”

“…Algorithmic modeling to predict or identify patients at high risk of overdose and other adverse drug-related (particularly opioid-related) outcomes has been increasingly used to direct overdose prevention interventions. Many investigators have used both conventional approaches (typically logistic regression) and more novel machine learning methods to predict risk of opioid-related harms, such as fatal and non-fatal overdose, opioid-related hospitalizations, incident opioid use disorder and persistent opioid use using large datasets including those from prescription drug monitoring programs [5][6][7][8][9], commercial claims and electronic health records [10][11][12][13][14][15][16][17], Medicare [18,19], Medicaid [20], the Veterans Administration [21,22] and other administrative claims [23].…”

Applying Machine Learning in Distributed Data Networks for Pharmacoepidemiologic and Pharmacovigilance Studies: Opportunities, Challenges, and Considerations

“…Similar to the use of supervised ML for diagnostic modeling in phenotyping activities, the use of supervised ML for prognostic modeling offers a data-adaptive approach for identifying potentially complex associations and interactions between a plethora of features to predict future health outcomes. Supervised ML has been used with electronic health data to create accurate risk prediction models for many health events for which prospective surveillance, prevention, early intervention, and advanced planning is invaluable, including opioid overdose [ 60 , 61 ], cancer-related mortality [ 62 ], suicidality [ 63 ], and high healthcare costs [ 64 ].…”