TAPER: Time-Aware Patient EHR Representation

Darabi, Sajad; Kachuee, Mohammad; Fazeli, Shayan; Sarrafzadeh, Majid

doi:10.1109/jbhi.2020.2984931

Cited by 53 publications

(20 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The attention weights are applied to calculate the likelihood of each medical code. Following the same idea, Darabi et al [23] learned the code representation with a Skip-gram model which is based on transformer network and trained a BERT model on the clinical notes leading to the results with time-stamps. The patient embedding obtained demonstrated the effectiveness of the utilization of the unstructured text data.…”

Section: Natural Language Processing In Medical Textmentioning

confidence: 99%

Multimodal temporal-clinical note network for mortality prediction

2021

View full text Add to dashboard Cite

Background Mortality prediction is an important task to achieve smart healthcare, especially for the management of intensive care unit. It can provide a reference for doctors to quickly predict the course of disease and customize early intervention programs for the patients in need. With the development of the electronic medical records, deep learning methods are introduced to deal with the prediction task. In the electronic medical records, clinical notes always contain rich and diverse medical information, including the clinical histories and reports during admission. Mortality prediction methods mostly rely on the temporal events such as medical examinations and ignore the related reports and history information in the clinical notes. We hope that we can utilize both temporal events and clinical notes information to get better mortality prediction results. Results We propose a multimodal temporal-clinical note network to model both temporal and clinical notes. Specifically, the clinical text are further processed for differentiating the chronic illness patients in the historical information of clinical notes from non-chronic illness patients. In order to further mine the information related to the mortality in the text, we learn the time series embedding with Long Short Term Memory networks and the clinical notes embedding with a label aware convolutional neural network. We also propose a scoring function to measure the importance of clinical note sections. Our approach achieved a better AUCPR and AUCROC than competing methods and visual explanations for word importance showed the interpretability improvement of the model. Conclusions We have tested our methodology on the MIMIC-III dataset. Contributions of different clinical note sections were uncovered by visualization methods. Our work demonstrates that the introduction of the medical history related information can improve the performance of the mortality prediction. Using label aware convolutional neural networks can further improve the results.

show abstract

Section: Natural Language Processing In Medical Textmentioning

confidence: 99%

Multimodal temporal-clinical note network for mortality prediction

2021

View full text Add to dashboard Cite

show abstract

“…This increased flexibility is particularly important when considering the temporal aspect of events given that the time and sequence of events can play a vital role when evaluating patient outcomes. Recent research has stressed the importance of temporal events in predictive analytics (30,31). Furthermore, data science techniques such as feature selection techniques can estimate and even select particular variables based upon predictive power.…”

Section: Background and Significancementioning

confidence: 99%

Evaluation of Machine Learning Techniques to Predict the Likelihood of Mental Health Conditions Following a First mTBI

et al. 2022

View full text Add to dashboard Cite

ObjectiveLimited research has evaluated the utility of machine learning models and longitudinal data from electronic health records (EHR) to forecast mental health outcomes following a traumatic brain injury (TBI). The objective of this study is to assess various data science and machine learning techniques and determine their efficacy in forecasting mental health (MH) conditions among active duty Service Members (SMs) following a first diagnosis of mild traumatic brain injury (mTBI).Materials and MethodsPatient demographics and encounter metadata of 35,451 active duty SMs who have sustained an initial mTBI, as documented within the EHR, were obtained. All encounter records from a year prior and post index mTBI date were collected. Patient demographics, ICD-9-CM and ICD-10 codes, enhanced diagnostic related groups, and other risk factors estimated from the year prior to index mTBI were utilized to develop a feature vector representative of each patient. To embed temporal information into the feature vector, various window configurations were devised. Finally, the presence or absence of mental health conditions post mTBI index date were used as the outcomes variable for the models.ResultsWhen evaluating the machine learning models, neural network techniques showed the best overall performance in identifying patients with new or persistent mental health conditions post mTBI. Various window configurations were tested and results show that dividing the observation window into three distinct date windows [−365:−30, −30:0, 0:14] provided the best performance. Overall, the models described in this paper identified the likelihood of developing MH conditions at [14:90] days post-mTBI with an accuracy of 88.2%, an AUC of 0.82, and AUC-PR of 0.66.DiscussionThrough the development and evaluation of different machine learning models we have validated the feasibility of designing algorithms to forecast the likelihood of developing mental health conditions after the first mTBI. Patient attributes including demographics, symptomatology, and other known risk factors proved to be effective features to employ when training ML models for mTBI patients. When patient attributes and features are estimated at different time window, the overall performance increase illustrating the importance of embedding temporal information into the models. The addition of temporal information not only improved model performance, but also increased interpretability and clinical utility.ConclusionPredictive analytics can be a valuable tool for understanding the effects of mTBI, particularly when identifying those individuals at risk of negative outcomes. The translation of these models from retrospective study into real-world validation models is imperative in the mitigation of negative outcomes with appropriate and timely interventions.

show abstract

“…In the irregular time interval domain, several studies have been conducted to apply self-attention to address different time intervals in a sequence [28,29]. The authors of [29] used a Transformer encoder, a symmetric time interval matrix, and a position matrix as inputs to consider irregular time in the recommendation system.…”

Section: Attention Mechanismmentioning

confidence: 99%

Time-Aware and Feature Similarity Self-Attention in Vessel Fuel Consumption Prediction

Park

Lee

Park³

et al. 2021

Applied Sciences

View full text Add to dashboard Cite

An accurate vessel fuel consumption prediction is essential for constructing a ship route network and vessel management, leading to efficient sailings. Besides, ship data from monitoring and sensing systems accelerate fuel consumption prediction research. However, the ship data consist of three properties: sequential, irregular time interval, and feature importance, making the predicting problem challenging. In this paper, we propose Time-aware Attention (TA) and Feature-similarity Attention (FA) applied to bi-directional Long Short-Term Memory (LSTM). TA acquires time importance by nonlinear function from irregular time intervals in each sequence and emphasizes data depending on the importance. FA emphasizes data based on similarities of features in the sequence by estimating feature importance with learnable parameters. Finally, we propose the ensemble model of TA and FA-based BiLSTM. The ensemble model, which consists of fully connected layers, is capable of simultaneously capturing different properties of ship data. The experimental results on ship data showed that the proposed model improves the performance in predicting fuel consumption. In addition to model performance, visualization results of attention maps and feature importance help to understand data properties and model characteristics.

show abstract

TAPER: Time-Aware Patient EHR Representation

Cited by 53 publications

References 34 publications

Multimodal temporal-clinical note network for mortality prediction

Multimodal temporal-clinical note network for mortality prediction

Evaluation of Machine Learning Techniques to Predict the Likelihood of Mental Health Conditions Following a First mTBI

Time-Aware and Feature Similarity Self-Attention in Vessel Fuel Consumption Prediction

Contact Info

Product

Resources

About