Artificial Intelligence Learning Semantics via External Resources for Classifying Diagnosis Codes in Discharge Notes

Abstract

BackgroundAutomated disease code classification using free-text medical information is important for public health surveillance. However, traditional natural language processing (NLP) pipelines are limited, so we propose a method combining word embedding with a convolutional neural network (CNN).ObjectiveOur objective was to compare the performance of traditional pipelines (NLP plus supervised machine learning models) with that of word embedding combined with a CNN in conducting a classification task identifying International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes in discharge notes.MethodsWe used 2 classification methods: (1) extracting from discharge notes some features (terms, n-gram phrases, and SNOMED CT categories) that we used to train a set of supervised machine learning models (support vector machine, random forests, and gradient boosting machine), and (2) building a feature matrix, by a pretrained word embedding model, that we used to train a CNN. We used these methods to identify the chapter-level ICD-10-CM diagnosis codes in a set of discharge notes. We conducted the evaluation using 103,390 discharge notes covering patients hospitalized from June 1, 2015 to January 31, 2017 in the Tri-Service General Hospital in Taipei, Taiwan. We used the receiver operating characteristic curve as an evaluation measure, and calculated the area under the curve (AUC) and F-measure as the global measure of effectiveness.ResultsIn 5-fold cross-validation tests, our method had a higher testing accuracy (mean AUC 0.9696; mean F-measure 0.9086) than traditional NLP-based approaches (mean AUC range 0.8183-0.9571; mean F-measure range 0.5050-0.8739). A real-world simulation that split the training sample and the testing sample by date verified this result (mean AUC 0.9645; mean F-measure 0.9003 using the proposed method). Further analysis showed that the convolutional layers of the CNN effectively identified a large number of keywords and automatically extracted enough concepts to predict the diagnosis codes.ConclusionsWord embedding combined with a CNN showed outstanding performance compared with traditional methods, needing very little data preprocessing. This shows that future studies will not be limited by incomplete dictionaries. A large amount of unstructured information from free-text medical writing will be extracted by automated approaches in the future, and we believe that the health care field is about to enter the age of big data.

Construction of a Semi-automatic ICD-10 Coding System

“…The technical requirements and computational cost are less than those of the other methods found in most studies [7], [11], [32][33][34][35][36]. Convolutional neural network (CNN) [18], [34][35][36] is one of the state of the art proposals to solve the problem of automatic ICD coding. Despite their high accuracy, there is still a long way to go before they can be used in practice.…”

“…Other techniques based on natural language processing (NLP) [13][14][15][16][17] can significantly increase the performance of automatic ICD coding by mapping the already assigned diagnoses of patients to ICD codes. Study [18] used word embedding combined with a convolutional neural network (CCN), which showed outstanding performance compared with the NLP plus supervised machine learning models. A prior study [19] automatically classify patients' diseases into an ICD-10-CM category based on the well known Web Ontology Language (OWL).…”

Construction of a semi-automatic ICD-10 coding system

“…The technical requirements and computational cost are less than those of the other methods found in most studies [7,11], [32][33][34][35][36]. CNN [18,[34][35][36] is one of the state of the art proposals to solve the problem of automatic ICD coding. Despite their high accuracy, there is still a long way to go before they can be used in practice.…”

“…Other techniques based on natural language processing (NLP) [13][14][15][16][17] can significantly increase the performance of automatic ICD coding by mapping the already assigned diagnoses of patients to ICD codes. Study [18] used word embedding combined with a convolutional neural network (CCN), which showed outstanding performance compared with the NLP plus supervised machine learning models. A prior study [19] automatically classify patients' diseases into an ICD-10-CM category based on the well-known Web Ontology Language.…”

Artificial Intelligence in Radiology Residency Training

“…23 For instance, AI algorithms were successful in identifying International Classification of Diseases, 10th Revision, diagnosis codes in a large set of hospital discharge notes, and algorithms were able to identify limb abnormalities (fractures and foreign bodies) in radiology reports, helping to reconcile report findings with appropriate care delivered by emergency department providers. 30,31 It is likely only a matter of time before AI is being used routinely to assist with radiology coding and billing. Some AI applications will likely have a more direct impact on improving resident training, for example, decision support algorithms, automated protocoling, and acute case triage algorithms, compared with others (►Table 1).…”