Unifying Heterogenous Electronic Health Records Systems via Text-Based Code Embedding: Study of Predictive Modeling (Preprint)

Abstract

BACKGROUND Substantial increase in the use of Electronic Health Records (EHRs) has opened new frontiers for predictive healthcare. However, while EHR systems are nearly ubiquitous, they lack a unified code system for representing medical concepts. Heterogeneous formats of EHR present a substantial barrier for the training and deployment of state-of-the-art deep learning models at scale. OBJECTIVE The aim of this study is to suggest a novel text embedding approach to overcome heterogeneity of EHR structure among different EHR systems. METHODS We introduce Description-based Embedding, DescEmb, a code-agnostic description-based representation learning framework for predictive modeling on EHR. DescEmb takes advantage of the flexibility of neural language understanding models while maintaining a neutral approach that can be combined with prior frameworks for task-specific representation learning or predictive modeling. RESULTS Based on five prediction tasks with two heterogeneous EHR datasets, DescEmb achieves comparable or superior performance to the traditional code-based embedding approach, especially under the zero-shot and few-shot transfer learning scenarios. We also demonstrate that DescEmb enables us to train a single model on a pooled dataset from heterogeneous EHR systems and achieve the same, if not better performance compared to training separate models for each EHR system. CONCLUSIONS Based on the promising results, we believe the description-based embedding approach on EHR will open a new direction for large-scale predictive modeling in healthcare.

GenHPF: General Healthcare Predictive Framework for Multi-Task Multi-Source Learning

“…DescEmb [25] proposed to resolve this problem by suggesting a text-based embedding, where hospital-specific feature values are first converted to textual descriptions (e.g., "401.9" → "unspecified essential hypertension"), then a text encoder paired with a sub-word tokenizer is used to obtain m i [37]. With Fig.…”

“…Additionally, in multi-source learning, our framework is not constrained by the features that are present in each schema since both the name n k i and the value v k i of the feature are used. A formal comparison of the conventional approach, DescEmb [25] and our approach for obtaining m i is provided below:…”

“…In another study, DescEmb [25] aimed to overcome the heterogeneity of medical codes by utilizing the clinical descriptions linked to each code, thereby partially enabling multi-source learning. Despite its text-based embedding to avoid the manual code mapping process, this approach still necessitates domain experts to conduct EHR system-specific preprocessing to select compatible and meaningful features from the EHRs.…”

Pre-training in Medical Data: A Survey

“…In conditions where there is a lack of data, it is possible to enhance the performance of the model [214] . The EHRsrelated tasks include prediction [33,126,[214][215][216][217][218][219][220][221][222] , information extraction from clinic notes [223][224][225][226] , the international classification of disease (ICD) coding [227,228] , medication recommendation [229,230] , etc.…”

“…Xu et al [216] introduced the medical knowledge graph combined with self-supervised pre-training to deal with the sparsity and high-dimensional issue of EHR data. Lu et al [221] utilised a pre-trained model to detect disease complications and compute the contributions of particular diseases and admissions. Using the self-supervised learning method, the pre-trained model was trained based on the hidden disease representation.…”