Word2Vec inversion and traditional text classifiers for phenotyping lupus

Abstract

BackgroundIdentifying patients with certain clinical criteria based on manual chart review of doctors’ notes is a daunting task given the massive amounts of text notes in the electronic health records (EHR). This task can be automated using text classifiers based on Natural Language Processing (NLP) techniques along with pattern recognition machine learning (ML) algorithms. The aim of this research is to evaluate the performance of traditional classifiers for identifying patients with Systemic Lupus Erythematosus (SLE) in comparison with a newer Bayesian word vector method.MethodsWe obtained clinical notes for patients with SLE diagnosis along with controls from the Rheumatology Clinic (662 total patients). Sparse bag-of-words (BOWs) and Unified Medical Language System (UMLS) Concept Unique Identifiers (CUIs) matrices were produced using NLP pipelines. These matrices were subjected to several different NLP classifiers: neural networks, random forests, naïve Bayes, support vector machines, and Word2Vec inversion, a Bayesian inversion method. Performance was measured by calculating accuracy and area under the Receiver Operating Characteristic (ROC) curve (AUC) of a cross-validated (CV) set and a separate testing set.ResultsWe calculated the accuracy of the ICD-9 billing codes as a baseline to be 90.00% with an AUC of 0.900, the shallow neural network with CUIs to be 92.10% with an AUC of 0.970, the random forest with BOWs to be 95.25% with an AUC of 0.994, the random forest with CUIs to be 95.00% with an AUC of 0.979, and the Word2Vec inversion to be 90.03% with an AUC of 0.905.ConclusionsOur results suggest that a shallow neural network with CUIs and random forests with both CUIs and BOWs are the best classifiers for this lupus phenotyping task. The Word2Vec inversion method failed to significantly beat the ICD-9 code classification, but yielded promising results. This method does not require explicit features and is more adaptable to non-binary classification tasks. The Word2Vec inversion is hypothesized to become more powerful with access to more data. Therefore, currently, the shallow neural networks and random forests are the desirable classifiers.

Quantitative assessment of informative immunophenotypic markers increases the diagnostic value of immunophenotyping in mature CD5‐positive B‐cell neoplasms

“…Machine learning methods based on predictive (classification) models using RF have recently been widely applied in many diagnostic, prognostic and therapeutic studies (40)(41)(42)(43)(44)(45)(46)(47)(48)(49)(50)(51)(52). Machine learning methods based on predictive (classification) models using RF have recently been widely applied in many diagnostic, prognostic and therapeutic studies (40)(41)(42)(43)(44)(45)(46)(47)(48)(49)(50)(51)(52).…”

“…Our study provides evidence for the clinical benefits of quantitative immunophenotyping by a rational and effective marker panel followed by the use of a predictive model (diagnostic classifier), minimizing the subjectivity of commonly used expert-based assessment. Machine learning methods based on predictive (classification) models using RF have recently been widely applied in many diagnostic, prognostic and therapeutic studies (40)(41)(42)(43)(44)(45)(46)(47)(48)(49)(50)(51)(52). Hereby we also showed its utility for the evaluation of flow cytometry data.…”

An application of machine learning to assist medication order review by pharmacists in a health care center

“…This technique has been applied to a variety of subjects in biomedical sciences with interesting results. [16][17][18][19][20][21][22] A hallmark feature of word2vec is that the resulting embeddings encode semantic relationships. A classic example would be that, in embeddings trained on an English language corpus, the vectors going from countries to their capital are similar.…”

Using Machine Learning to Identify Health Outcomes from Electronic Health Record Data

“…Turner et al . (37) used an NLP system involving machine learning to identify patients with SLE from outpatient clinical notes. Similar to the previous example for falls (32), the authors extracted medical concepts from clinical notes and used them as inputs to one of four classifiers: a shallow neural network, random forest, naïve Bayes, or support vector machine.…”

“…Turner et al (37) also explored the use of a more contemporary machine learning-based NLP approach to identify patients with SLE from clinical notes. This approach involved using the increasingly popular Word2Vec system (39) to create features from words in the text using word embeddings.…”