Natural language processing for the assessment of cardiovascular disease comorbidities: The cardio‐Canary comorbidity project

Abstract

Objective: Accurate ascertainment of comorbidities is paramount in clinical research. While manual adjudication is labor‐intensive and expensive, the adoption of electronic health records enables computational analysis of free‐text documentation using natural language processing (NLP) tools. Hypothesis: We sought to develop highly accurate NLP modules to assess for the presence of five key cardiovascular comorbidities in a large electronic health record system. Methods: One‐thousand clinical notes were randomly selected from a cardiovascular registry at Mass General Brigham. Trained physicians manually adjudicated these notes for the following five diagnostic comorbidities: hypertension, dyslipidemia, diabetes, coronary artery disease, and stroke/transient ischemic attack. Using the open‐source Canary NLP system, five separate NLP modules were designed based on 800 “training‐set” notes and validated on 200 “test‐set” notes. Results: Across the five NLP modules, the sentence‐level and note‐level sensitivity, specificity, and positive predictive value was always greater than 85% and was most often greater than 90%. Accuracy tended to be highest for conditions with greater diagnostic clarity (e.g. diabetes and hypertension) and slightly lower for conditions whose greater diagnostic challenges (e.g. myocardial infarction and embolic stroke) may lead to less definitive documentation. Conclusion: We designed five open‐source and highly accurate NLP modules that can be used to assess for the presence of important cardiovascular comorbidities in free‐text health records. These modules have been placed in the public domain and can be used for clinical research, trial recruitment and population management at any institution as well as serve as the basis for further development of cardiovascular NLP tools.

Collaborative and privacy-enhancing workflows on a clinical data warehouse: an example developing natural language processing pipelines to detect medical conditions

“…The performances of our main NLP-ML-CLINICAL pipeline are comparable to the best published results regarding similar approaches, although slightly lower than those reached by Singh et al 13 , 15 , 37 We emphasize that these studies remain difficult to compare as they focus on different languages (French vs English) and texts (eg, Singh et al consider only medical and surgical history sections). Aggregating extractions from the entity level to the stay level significantly improves performances, which is a known and notable result as stay-level or patient-level features are often of higher importance and interest than entity-level features, eg, in epidemiological studies.…”

“…Aggregating extractions from the entity level to the stay level significantly improves performances, which is a known and notable result as stay-level or patient-level features are often of higher importance and interest than entity-level features, eg, in epidemiological studies. 15 Notably, sensitivity is greatly improved, as the aggregation step allows for missed entities of a specific condition to be compensated by other occurrences of the same condition. Similarly, conditions with 2 levels of severity benefit from this aggregation since severity would not necessarily be mentioned on each entity.…”

Collaborative and privacy-preserving workflows on a clinical data warehouse: an example developing natural language processing pipelines to detect medical conditions

“…The performances of our main NLP-ML-CLINICAL pipeline are comparable to the best published results regarding similar algorithms although slightly lower than those reached by Singh et al . [13,15,36] We emphasize that these studies remain difficult to compare as they focus on different languages (French vs. English) and texts (e.g., Singh et al . consider only medical and surgical history sections).…”

“…Nevertheless, it was shown that information could be efficiently obtained from clinical notes using Natural Language Processing (NLP) algorithms instead, those algorithms relying even more on machine learning (ML) techniques such as language models. [12][13][14][15][16][17][18][19] Developing tools to this end remains challenging, and many difficulties are yet to be overcome for a wide community to benefit from them. [4,7,[19][20][21][22][23][24] First, the optimal NLP technologies are still debated.…”

Using natural language processing for automated classification of disease and to identify misclassified ICD codes in cardiac disease

“…Such techniques offer promising solutions for aiding in the often still manual and labour-intensive process of ICD coding, and for correcting ICD-code related errors. 6 …”