Addressing Human Subjectivity via Transfer Learning: An Application to Predicting Disease Outcome in Multiple Sclerosis Patients

Abstract

Predicting disease course is critical in chronic progressive diseases such as multiple sclerosis (MS). In our work we are applying machine learning methods to longitudinal records of MS patients to build a classifier that predicts whether a patient will have a significant increase in disability at the five year mark using information from the first two years of clinical visits. This prediction is key for choosing among the available treatments as some have more troubling side-effect profiles. Two challenges arise while learning with this data. First, patient data involves the physician's (possibly subjective) evaluation. Because a patient's data may come from one doctor on the first visit and different doctors on subsequent visits, it can be difficult to form an accurate predictor of disease outcome. In particular, some physicians may be biased in one direction, scoring each patient as more severe than would other physicians, while others may be biased in the opposite direction. Another challenge is that it is much easier to classify the cases with low future disability compared to cases with high future disability at early stages of the disease. This asymmetric property is due to the nature of the disease rather than class imbalance. In this paper we introduce a new transfer learning approach to handle these challenges. The algorithm builds a single SVM classifier for each doctor by dividing the entire dataset into primary (instances from the doctor) and auxiliary (instances from other doctors) sets. When applied to our dataset of MS patients our new approach is able to realize a significant increase in prediction performance over approaches that form a single SVM classifier for the entire dataset or for the physician's dataset alone.

A Calibrated Ensemble Algorithm to Address Data Heterogeneity in Machine Learning: An Application to Identify Severe SLE Flares in Lupus Patients

“…Addressing data heterogeneity in machine learning is an active research area because data collected from a complex real-world environment hardly follows a single underlying distribution. As alluded in Section I, researchers have resorted to techniques including domain knowledge integration [4], [11], multi-view learning (MVL) [6], [12], multi-task learning (MTL) [5], [13], and transfer learning (TL) [3], [7].…”

“…In the MTL and TL domain, Hu et al applied transfer learning to generate individualized patient models, grounded in the wealth of population data, while also detecting and adjusting for inter-patient variabilities based on each patient's own histologic data [13]. Zhao et al [3] applied transfer learning techniques to address human subjectivity in predicting disease course for chronic progressive diseases.…”

“…Practitioners applying machine learning to real-world data often find themselves in a common predicament: data are collected from heterogeneous sources and, consequently, have different underlying distributions. This is particularly true in the medical domain where patients may belong to distinctive subgroups with altered disease characteristics, or different physicians can introduce biases due to subjective interpretations of the clinical test or lab results [3], [4].…”