Using deep-learning to obtain calibrated individual disease and ADL damage transition probabilities between successive ELSA waves

Abstract

We predictively model damage transition probabilities for binary health outputs of 19 diseases and 25 activities of daily living states (ADLs) between successive waves of the English Longitudinal Study of Aging (ELSA). Model selection between deep neural networks (DNN), random forests, and logistic regression found that a simple one-hidden layer 128-node DNN was best able to predict future health states (AUC ≥ 0.91) and average damage probabilities (R^2 ≥ 0.92). Feature selection from 134 explanatory variables found that 33 variables are sufficient to predict all disease and ADL states well. Deciles of predicted damage transition probabilities were well calibrated, but correlations between predicted health states were stronger than observed. The hazard ratios (HRs) between high-risk deciles and the average were between 3 and 10; high prevalence damage transitions typically had smaller HRs. Model predictions were good across all individual ages. A simple one-hidden layer DNN predicts multiple binary diseases and ADLs with well calibrated damage and repair transition probabilities.