Instance Weighting for Patient-Specific Risk Stratification Models

Abstract

Accurate risk models for adverse outcomes can provide important input to clinical decision-making. Surprisingly, one of the main challenges when using machine learning to build clinically useful risk models is the small amount of data available. Risk models need to be developed for specific patient populations, specific institutions, specific procedures, and specific outcomes. With each exclusion criterion, the amount of relevant training data decreases, until there is often an insufficient amount to learn an accurate model. This difficulty is compounded by the large class imbalance that is often present in medical applications.In this paper, we present an approach to address the problem of small data using transfer learning methods in the context of developing risk models for cardiac surgeries. We explore ways to build surgery-specific and hospital-specific models (the target task) using information from other kinds of surgeries and other hospitals (source tasks). We propose a novel method to weight examples based on their similarity to the target task training examples to take advantage of the useful examples while discounting less relevant ones.We show that incorporating appropriate source data in training can lead to improved performance over using only target task training data, and that our method of instance weighting can lead to further improvements. Applied to a surgical risk stratification task, our method, which used data from two institutions, performed comparably to the risk model published by the Society for Thoracic Surgeons, which was developed and tested on over one hundred thousand surgeries from hundreds of institutions.

A Roadmap for Optimizing Asthma Care Management via Computational Approaches

“…Different systems have differing data distributions [ 53 ] and collected attributes, impacting the performance and applicability of a model trained using one system’s data for another system [ 54 - 57 ]. To address these two issues, one can perform transfer learning and use other source systems’ information to improve model accuracy for the target system [ 54 , 58 , 59 ]. Transfer learning typically requires using other source systems’ raw data [ 60 , 61 ].…”

Dynamic Prediction of ICU Mortality Risk Using Domain Adaptation

“…Likewise, authors in [24] proposed an approach that models the mortality probability as a latent state that evolves over time. Authors in [25] proposed an approach to address the problem of small data using transfer learning in the context of developing risk models for cardiac surgeries. They explored ways to build surgeryspecific and hospital-specific models using information from other kinds of surgeries and other hospitals.…”

“…Following [25], in this work we use feature transference, but in a quite different way, as follows: (i) instead of applying instance weighting, we employed a deep model that transfers domain-shared features; (ii) we studied a broader scenario that includes diverse ICU domains; and (iii) our models employ both spatial and temporal feature extraction, being able to predict patient outcomes dynamically.…”

Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting

“…15 Previously, transfer learning has been applied in the health care setting for segmenting magnetic resonance images across multiple scanners and postprocessing modes, 16 improving Clostridium difficile infection prediction performance using cross-institutional data, 17 creating time-varying risk assessment methods from EHR data, 18 and assessing the risk of adverse events in cardiac surgery. 19 However, there has been limited 2 Biomedical Informatics Insights work toward minimizing the data used in creating effective clinical decision support tools. 19 In this study, we build on our prior machine learning work in developing AutoTriage, an endto-end EHR-based algorithmic analysis system for forecasting patient mortality or predicting the results of intrahospital ward transfer and/patient discharge.…”

“…19 However, there has been limited work toward minimizing the data used in creating effective clinical decision support tools. 19 In this study, we build on our prior machine learning work in developing AutoTriage , an end-to-end EHR-based algorithmic analysis system for forecasting patient mortality or predicting the results of intrahospital ward transfer and/patient discharge. For mortality prediction, we previously demonstrated that AutoTriage outperformed canonical risk scoring systems, SAPS II (the Simplified Acute Physiology Score 20 ), SOFA, MEWS, 6 and similarly outperformed MET and MEWS for discharge prediction.…”