Identifying factors associated with opioid cessation in a biracial sample using machine learning

Abstract

LASSO implemented in the R package 'glmnet' (1) was used for both feature selection and prediction. The shrinkage parameter lambda in the penalty term of LASSO regression was obtained using 10-fold cross validation on the training set 10 times. Separate accuracy criteria of either misclassification error or AUC were used to search for the lambda with the best model fit. The "1SE rule (2)" which aims to find the simplest model with comparable accuracy to the best model, was used to identify lambda whose cross validation error was one standard error unit from the lowest cross validation error on the training set. We identified the significant features by fitting lambdas on the training set. The test set accuracy was evaluated by using the class probability prediction of test set as an input to Scikit-learn (3) (roc_auc_score and f1_score) to obtain our final accuracy measures: F1 score and AUC. Support Vector Machine (SVM) with Recursive Feature EliminationLinear SVM with recursive feature elimination was implemented using the Scikit-learn python package. A soft margin was included in the model to reduce overfitting. Briefly, we set the penalty parameter C by exhaustively searching the range of values between 2 -20 and 2 8 that covers the range recommended by Hsu et.al (4) using 10-fold cross validation with balanced class weights on the training set . We applied the parameter C chosen by either maximizing AUC or F1 score (whichever yielded the highest accuracy) in the training set for recursive feature selection. A feature's importance, represented by its weight, was used as the input to the recursive feature elimination function, where 10-fold cross validation was used to find the combination of features that maximizes either AUC or F1 score (whichever yielded the highest accuracy). Selected features from each model were applied to the test set to obtain the overall test accuracy.