Interpreting patient-Specific risk prediction using contextual decomposition of BiLSTMs: application to children with asthma

Abstract

BackgroundPredictive modeling with longitudinal electronic health record (EHR) data offers great promise for accelerating personalized medicine and better informs clinical decision-making. Recently, deep learning models have achieved state-of-the-art performance for many healthcare prediction tasks. However, deep models lack interpretability, which is integral to successful decision-making and can lead to better patient care. In this paper, we build upon the contextual decomposition (CD) method, an algorithm for producing importance scores from long short-term memory networks (LSTMs). We extend the method to bidirectional LSTMs (BiLSTMs) and use it in the context of predicting future clinical outcomes using patients’ EHR historical visits.MethodsWe use a real EHR dataset comprising 11071 patients, to evaluate and compare CD interpretations from LSTM and BiLSTM models. First, we train LSTM and BiLSTM models for the task of predicting which pre-school children with respiratory system-related complications will have asthma at school-age. After that, we conduct quantitative and qualitative analysis to evaluate the CD interpretations produced by the contextual decomposition of the trained models. In addition, we develop an interactive visualization to demonstrate the utility of CD scores in explaining predicted outcomes.ResultsOur experimental evaluation demonstrate that whenever a clear visit-level pattern exists, the models learn that pattern and the contextual decomposition can appropriately attribute the prediction to the correct pattern. In addition, the results confirm that the CD scores agree to a large extent with the importance scores generated using logistic regression coefficients. Our main insight was that rather than interpreting the attribution of individual visits to the predicted outcome, we could instead attribute a model’s prediction to a group of visits.ConclusionWe presented a quantitative and qualitative evidence that CD interpretations can explain patient-specific predictions using CD attributions of individual visits or a group of visits.

Importance of GWAS Risk Loci and Clinical Data in Predicting Asthma Using Machine-Learning Approaches

“…The accuracy of our study was lower than that of AlSaad et al [33] probably because the sample size of our study was relatively small, and we selected fewer clinical data. Collecting the clinical data for control samples was di cult, particularly for the clinical indicators which were not checked.…”

“…Introducing clinical data as environmental exposure factors was necessary to improve the accuracy of asthma-prediction models. AlSaad et al used a real electronic health record dataset comprising 6159 asthma cases and 4912 controls to predict the risk of asthma and obtained the highest AUC of 0.831 [33]. In contrary to using GWAS risk loci or clinical data alone, we combined GWAS risk loci and clinical data and obtained a more accurate asthma-prediction models with an AUC of 79.7%.…”

Comparison of Neural Language Modeling Pipelines for Outcome Prediction From Unstructured Medical Text Notes

“…In NLP, deep learning methods such as Long Short-Term Memory (LSTM) and its variants use a preprocessing pipeline that includes a filtering process based on predefined controlled vocabulary terms before transforming data into training vectors [15], [16].In this recent study [17], the authors propose an online medical pre-diagnosis support in which semantic and sequential features are extracted from a patient's inputs using a CNN-RNN-based architecture model to predict a diagnosis.…”

Explaining artificial intelligence with visual analytics in healthcare

“…(1) Nauta et al (2020) globally explain how an ANN predicts coma outcome: for a fixed epoch and a fixed hidden layer, all input activations are projected onto a scatter plot, and users can then select clusters to train a decision tree that distinguishes them. (2) Alsaad et al (2019) use contextual decomposition to locally explain how a long short-term memory (LSTM) network predicts asthma based on clinical visits: each visit's contribution to the prediction is visualized in a heat map matrix that also highlights the most predictive subset of visits.…”