Prediction of premature all-cause mortality: A prospective general population cohort study comparing machine-learning and standard epidemiological approaches

Abstract

Background Prognostic modelling using standard methods is well-established, particularly for predicting risk of single diseases. Machine-learning may offer potential to explore outcomes of even greater complexity, such as premature death. This study aimed to develop novel prediction algorithms using machine-learning, in addition to standard survival modelling, to predict premature all-cause mortality. Methods A prospective population cohort of 502,628 participants aged 40–69 years were recruited to the UK Biobank from 2006–2010 and followed-up until 2016. Participants were assessed on a range of demographic, biometric, clinical and lifestyle factors. Mortality data by ICD-10 were obtained from linkage to Office of National Statistics. Models were developed using deep learning, random forest and Cox regression. Calibration was assessed by comparing observed to predicted risks; and discrimination by area under the ‘receiver operating curve’ (AUC). Findings 14,418 deaths (2.9%) occurred over a total follow-up time of 3,508,454 person-years. A simple age and gender Cox model was the least predictive (AUC 0.689, 95% CI 0.681–0.699). A multivariate Cox regression model significantly improved discrimination by 6.2% (AUC 0.751, 95% CI 0.748–0.767). The application of machine-learning algorithms further improved discrimination by 3.2% using random forest (AUC 0.783, 95% CI 0.776–0.791) and 3.9% using deep learning (AUC 0.790, 95% CI 0.783–0.797). These ML algorithms improved discrimination by 9.4% and 10.1% respectively from a simple age and gender Cox regression model. Random forest and deep learning achieved similar levels of discrimination with no significant difference. Machine-learning algorithms were well-calibrated, while Cox regression models consistently over-predicted risk. Conclusions Machine-learning significantly improved accuracy of prediction of premature all-cause mortality in this middle-aged population, compared to standard methods. This study illustrates the value of machine-learning for risk prediction within a traditional epidemiological study design, and how this approach might be reported to assist scientific verification.

In praise of partially interpretable predictors

“…In answering this question, one is led to conclude that the physical justifications for models may seem valid but often not to the exactitude or precision required especially when good prediction is a goal. This is seen in Weng et al and more generally is consistent with Milkowski et al (and the references therein). In addition, analysts and experimentalists alike want to avoid being misled by insisting on a level of interpretability that cannot be justified.…”

“…Indeed, another example of this phenomenon can be found in Weng et al . They compared Cox models (highly interpretable) with both a deep learning technique and random forests (not as interpretable) for predicting human lifespan.…”

“…The caveats on this intuition include that the system under study be sufficiently complex, the modeling be not too misleading, and the sample size be in the right range relative to the complexity of the predictors and data generator, among others. Indeed, another example of this phenomenon can be found in Weng et al [33]. They compared Cox models (highly interpretable) with both a deep learning technique and random forests (not as interpretable) for predicting human lifespan.…”

Prediction of comorbid anxiety and depression using machine learning models in cancer survivors

“…Future studies would possibly show a better performance if a single machine learning algorithm is combined with various algorithms rather than a single algorithm model. Machine learning were considered as "black-box", and had limitations to reveal how features are interacting and what effect they have on the outcome (15,25). However, the feature importance provided information evaluate the importance ranking of each factor in the classi cation decision.…”

A study protocol for a predictive algorithm to assess population-based premature mortality risk: Premature Mortality Population Risk Tool (PreMPoRT)

“…To date, the majority of prediction models have focused on all-cause mortality [ 28 ], all-cause mortality in defined population subgroups (i.e., infant mortality, maternal mortality, trauma patients) [ 29 – 31 ], or use data sources (i.e., electronic health records, biological specimens) that are not publicly available [ 32 ]. To our knowledge, no population-level risk prediction algorithm, using routinely collected public available data, has been developed for premature mortality.…”