De-identification of clinical notes via recurrent neural network and conditional random field

Abstract

De-identification, identifying information from data, such as protected health information (PHI) present in clinical data, is a critical step to enable data to be shared or published. The 2016 Centers of Excellence in Genomic Science (CEGS) Neuropsychiatric Genome-scale and RDOC Individualized Domains (N-GRID) clinical natural language processing (NLP) challenge contains a de-identification track in de-identifying electronic medical records (EMRs) (i.e., track 1). The challenge organizers provide 1000 annotated mental health records for this track, 600 out of which are used as a training set and 400 as a test set. We develop a hybrid system for the de-identification task on the training set. Firstly, four individual subsystems, that is, a subsystem based on bidirectional LSTM (long-short term memory, a variant of recurrent neural network), a subsystem-based on bidirectional LSTM with features, a subsystem based on conditional random field (CRF) and a rule-based subsystem, are used to identify PHI instances. Then, an ensemble learning-based classifiers is deployed to combine all PHI instances predicted by above three machine learning-based subsystems. Finally, the results of the ensemble learning-based classifier and the rule-based subsystem are merged together. Experiments conducted on the official test set show that our system achieves the highest micro F1-scores of 93.07%, 91.43% and 95.23% under the “token”, “strict” and “binary token” criteria respectively, ranking first in the 2016 CEGS N-GRID NLP challenge. In addition, on the dataset of 2014 i2b2 NLP challenge, our system achieves the highest micro F1-scores of 96.98%, 95.11% and 98.28% under the “token”, “strict” and “binary token” criteria respectively, outperforming other state-of-the-art systems. All these experiments prove the effectiveness of our proposed method.

A review of Automatic end-to-end De-Identification: Is High Accuracy the Only Metric?

“…De-identification system Machine learning S1 (Zhao, Zhang, Ma, and Li (2018)), S2 (Chen, Cullen, and Godwin (2015)) S3 (Dernoncourt, Lee, Uzuner, and Szolovits (2017)), S4 (Yadav, Ekbal, Saha, Pathak, and Bhattacharyya (2017)), S5 ), S6 ) Hybrid S7 (Yang and Garibaldi (2015)) S8 (Liu, Tang, Wang, and Chen (2017)) S9 (Lee, Dernoncourt, Uzuner, and Szolovits (2016)) S10 (Dehghan, Kovacevic, Karystianis, Keane, and Nenadic (2015)) S11 (Yang and Garibaldi (2015)) S12 (He, Guan, Cheng, Cen, and Hua (2015)) S13 (Liu, Chen, Tang, Wang, Chen, Li, Wang, Deng, and Zhu (2015)) S14 (Phuong and Chau (2016)) S15 (Bui, Wyatt, and Cimino (2017a)) S16 (Jiang, Zhao, He, Guan, and Jiang (2017)) S17 (Lee, Wu, Zhang, Xu, Xu, and Roberts (2017)) S18 (Shweta, Kumar, Ekbal, Saha, and Bhattacharyya (2016)) In this section, we outline the most significant achievement of automating end-toend de-identification system: improving accuracy. It has been argued that as far as de-identification is concerned, perfection cannot be achieved; however, 95% accuracy is considered to be the rule of thumb and universally accepted value ; ).…”

Audio De-identification - a New Entity Recognition Task

“…The M I2B2 model is tuned to achieve state-ofthe-art results on textual medical notes, such as in Dernoncourt et al (2017a); Liu et al (2017). It should be stressed that the model was used as is, without an attempt to adapt it to the domain of ASR output.…”

Machine learning in mental health: A systematic scoping review of methods and applications

“…Deep Learning (word2vec) [299] Research Articles [299] Depression DT [303], kNN [134,298], NN [295], Regression [294,296], RF [134], SVM [134], Linear Discriminant Analysis [134] Survey [296,303,304], Social Media [298], Electronic Health Records [295], Imaging [134,294], Biological [134,296] Healthy Ageing RF [304] Survey [304] Psychosis SVM, Multiple Kernel Learning [297] Imaging [297] Schizophrenia RF [291], SVM [291,293], Linear Discriminant Analysis [291], kNN [291] Insurance [291], Imaging [293] Substance Use Topic modelling [306] Interview [306] Symptom Severity NN [301] Clinical Notes [301] Wellbeing BN [302], SVM [302], Deep Learning (paragraph2vec) [300], NN [307] Clinical Notes [300,302] As an emerging field, there are understandably significant gaps for future research to address. It is evident that the majority of papers focus on diagnosis and detection, particularly on depression, suicide risk and cognitive decline.…”