Automated detection of substance use information from electronic health records for a pediatric population

Abstract

Objective Substance use screening in adolescence is unstandardized and often documented in clinical notes, rather than in structured electronic health records (EHRs). The objective of this study was to integrate logic rules with state-of-the-art natural language processing (NLP) and machine learning technologies to detect substance use information from both structured and unstructured EHR data. Materials and Methods Pediatric patients (10-20 years of age) with any encounter between July 1, 2012, and October 31, 2017, were included (n = 3890 patients; 19 478 encounters). EHR data were extracted at each encounter, manually reviewed for substance use (alcohol, tobacco, marijuana, opiate, any use), and coded as lifetime use, current use, or family use. Logic rules mapped structured EHR indicators to screening results. A knowledge-based NLP system and a deep learning model detected substance use information from unstructured clinical narratives. System performance was evaluated using positive predictive value, sensitivity, negative predictive value, specificity, and area under the receiver-operating characteristic curve (AUC). Results The dataset included 17 235 structured indicators and 27 141 clinical narratives. Manual review of clinical narratives captured 94.0% of positive screening results, while structured EHR data captured 22.0%. Logic rules detected screening results from structured data with 1.0 and 0.99 for sensitivity and specificity, respectively. The knowledge-based system detected substance use information from clinical narratives with 0.86, 0.79, and 0.88 for AUC, sensitivity, and specificity, respectively. The deep learning model further improved detection capacity, achieving 0.88, 0.81, and 0.85 for AUC, sensitivity, and specificity, respectively. Finally, integrating predictions from structured and unstructured data achieved high detection capacity across all cases (0.96, 0.85, and 0.87 for AUC, sensitivity, and specificity, respectively). Conclusions It is feasible to detect substance use screening and results among pediatric patients using logic rules, NLP, and machine learning technologies.

Machine Learning Approaches for Electronic Health Records Phenotyping: A Methodical Review

“…Extracted medical concepts are often engineered into features for traditional ML-based phenotyping models (eg. number of positive mentions of the phenotype in a patient record) [30,33,43,81–85]. The most common clinical NLP software was clinical Text Analysis and Knowledge Extraction System (cTAKES) and was used in 20% of the articles [86].…”

“…Extracted medical concepts are often engineered into features for traditional ML-based phenotyping models (eg. number of positive mentions of the phenotype in a patient record) [30,33,43,[81][82][83][84][85].…”

“…number of billing codes and free-text mentions of the phenotype in a patient record) and (ii) the 'gold-standard' phenotype labels (eg.presence/absence of the phenotype). Traditionally, gold-standard labels are annotated by manual review of patient records [37,85,116]. Labels have also been derived from registry data [33], laboratory results [61,112,117], diagnosis codes [30,57,58,[118][119][120], and rule-based algorithms [59,[121][122][123] to enable more rapid development of labeled datasets.…”

“…33 of the 72 supervised learning articles employ deep learning, with 31 of these articles using word embeddings as the input layer. Feed-forward neural networks are the least frequently used architecture [25,32,108,111,118,133], while recurrent neural networks (RNNS), [27,34,59,69,[74][75][76]85,90,94,95,103,[105][106][107]109,110,123,126,130,134], convolutional neural networks (CNNs) [33,75,83,91,93,94,103,106,107,112,126,[134][135][136], and their variants are frequently used (Supplementary Material Table S12). RNNs are particularly well suited for sequential data from longitudinal EHRs.…”

“…For example, Martin et al found that a penalized elastic net logistic regression consistently outperformed FFNNs for identifying frailty aspects such as musculoskeletal problems and respiratory impairment (F1-score Elastic net 0.51-0.65, F1-score FFNN 0.24-0.62) [95]. Several authors also commented on challenges related to model interpretability and transportability of deep learning models across sites [25,85,91,130].…”