Safe opioid prescribing: a prognostic machine learning approach to predicting 30-day risk after an opioid dispensation in Alberta, Canada

Abstract

ObjectiveTo develop machine learning models employing administrative health data that can estimate risk of adverse outcomes within 30 days of an opioid dispensation for use by health departments or prescription monitoring programmes.Design, setting and participantsThis prognostic study was conducted in Alberta, Canada between 2017 and 2018. Participants included all patients 18 years of age and older who received at least one opioid dispensation. Pregnant and cancer patients were excluded.ExposureEach opioid dispensation served as an exposure.Main outcomes/measuresOpioid-related adverse outcomes were identified from linked administrative health data. Machine learning algorithms were trained using 2017 data to predict risk of hospitalisation, emergency department visit and mortality within 30 days of an opioid dispensation. Two validation sets, using 2017 and 2018 data, were used to evaluate model performance. Model discrimination and calibration performance were assessed for all patients and those at higher risk. Machine learning discrimination was compared with current opioid guidelines.ResultsParticipants in the 2017 training set (n=275 150) and validation set (n=117 829) had similar baseline characteristics. In the 2017 validation set, c-statistics for the XGBoost, logistic regression and neural network classifiers were 0.87, 0.87 and 0.80, respectively. In the 2018 validation set (n=393 023), the corresponding c-statistics were 0.88, 0.88 and 0.82. C-statistics from the Canadian guidelines ranged from 0.54 to 0.69 while the US guidelines ranged from 0.50 to 0.62. The top five percentile of predicted risk for the XGBoost and logistic regression classifiers captured 42% of all events and translated into post-test probabilities of 13.38% and 13.45%, respectively, up from the pretest probability of 1.6%.ConclusionMachine learning classifiers, especially incorporating hospitalisation/physician claims data, have better predictive performance compared with guideline or prescription history only approaches when predicting 30-day risk of adverse outcomes. Prescription monitoring programmes and health departments with access to administrative data can use machine learning classifiers to effectively identify those at higher risk compared with current guideline-based approaches.

Using machine learning to study the effect of medication adherence in Opioid Use Disorder

“…[ 19 ] aimed to predict opioid-related overdose using statewide prescription drug monitoring program and hospital discharge data in the US state of Tennessee. [ 20 ], using data from Alberta, Canada, aimed to predict overdose risk within 30 days after an opioid dispensation on the basis of features related to opioid dispensations. Their cohorts are thus different from ours, and they do not look at MOUD adherence, the focus of our study.…”

“…Machine learning techniques have demonstrated superior performance in the prediction of unobserved events in some contexts and are gaining increasing acceptance in medical and related fields [ 10 – 12 ]. In the context of OUD, researchers have recently used machine learning to predict overdose outcomes, for the most part based on prior opioid prescriptions [ 13 – 20 ]; these approaches focus on factors prior to and at the time of diagnosis, and do not consider the ongoing interaction between patient and treatment that can be crucial in longer term outcomes like medication adherence. Moreover, restricting modeling to cases involving prior opioid prescriptions becomes less helpful as many with OUD turn to illicit sources of opioids due to more rigorous prescribing controls and prescriber awareness of the dangers of over-prescribing prescription opioids.…”

Development and Validation of a Machine Learning Model to Estimate Risk of Adverse Outcomes Within 30 Days of Opioid Dispensation

“…This study used a supervised ML approach that trained an XGBoost model on opioid dispensations between January 1, 2018, and December 31, 2019, in Alberta, Canada. We used XGBoost due to it producing the highest prediction performance based on our previous work 14 and also because it generates an interpretable model. We included all Albertans who were 18 years or older and had received at least 1 opioid dispensation from a community pharmacy.…”

“…This study used a supervised ML approach that trained an XGBoost model on opioid dispensations between January 1, 2018, and December 31, 2019, in Alberta, Canada. We used XGBoost due to it producing the highest prediction performance based on our previous work 14 and also because it…”

“…Building on our previous work, 14 the objective of this study was to develop and validate a proof-of-concept XGBoost 15 , 16 ML model for use by the CPSA that could estimate the risk of adverse outcomes within 30 days of an opioid dispensation in Alberta, Canada. The ML model was trained only on prescription drug records to simulate a potential ML classifier deployment by the CPSA that is aligned with the type of data to which TPP Alberta has access.…”