The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up

Abstract

Accurate prediction of progression in subjects at risk of Alzheimer's disease is crucial for enrolling the right subjects in clinical trials. However, a prospective comparison of state-of-the-art algorithms for predicting disease onset and progression is currently lacking. We present the findings of The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge, which compared the performance of 92 algorithms from 33 international teams at predicting the future trajectory of 219 individuals at risk of Alzheimer's disease. Challenge participants were required to make a prediction, for each month of a 5-year future time period, of three key outcomes: clinical diagnosis, Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog13), and total volume of the ventricles. No single submission was best at predicting all three outcomes. For clinical diagnosis and ventricle volume prediction, the best algorithms strongly outperform simple baselines in predictive ability. However, for ADAS-Cog13 no single submitted prediction method was significantly better than random guessing. On a limited, cross-sectional subset of the data emulating clinical trials, performance of best algorithms at predicting clinical diagnosis decreased only slightly (3% error increase) compared to the full longitudinal dataset. Two ensemble methods based on taking the mean and median over all predictions, obtained top scores on almost all tasks. Better than average performance at diagnosis prediction was generally associated with the additional inclusion of features from cerebrospinal fluid (CSF) samples and diffusion tensor imaging (DTI). On the other hand, better performance at ventricle volume prediction was associated with inclusion of summary statistics, such as patient-specific biomarker trends. The submission system remains open via the website https://tadpole.grand-challenge.org, while code for submissions is being collated by TADPOLE SHARE: https://tadpole-share.github.io/. Our work suggests that current prediction algorithms are accurate for biomarkers related to clinical diagnosis and ventricle volume, opening up the possibility of cohort refinement in clinical trials for Alzheimer's disease.

Multi-task longitudinal forecasting with missing values on Alzheimer's Disease

“…As in the TADPOLE competition [28,47], we consider: the clinical diagnosis (NC, MCI, AD) denoted as D, the ventricle volume of the MRI data denoted as V and the ADAS-cog 13 score denoted as A.…”

Ten years of image analysis and machine learning competitions in dementia

“…• The Alzheimer's Disease Prediction Of Longitudinal Evolution Challenge (TADPOLE, 2017) 6 : the challenge consisted of three tasks in predicting future evolution of individuals at risk for Alzheimer's disease (Marinescu et al, 2018(Marinescu et al, , 2022. These tasks were to predict three key outcomes: 1) clinical diagnosis, 2) Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog13), and 3) total volume of the ventricles.…”

“…The only method that performed better than random guessing was a simple mixed effects model. According to Marinescu et al (2022), the difficulty could be due to variability in administering the cognitive tests, practice effects and a short follow-up time.…”

“…The method that achieved best accuracy used an ensemble of random forest classifiers and morphological MRI features. TADPOLE (Marinescu et al, 2022) What are the data, features and approaches that are the most predictive of future progression of subjects at risk of Alzheimer's disease to aid identification of patients for inclusion in clinical trials who are in early stages of disease and are likely to progress over the short-tomedium term (1-5 years)?…”

Evaluation of machine learning algorithms for Health and Wellness applications: a tutorial

“…Especially, if a clinical applicability of a trained machine learning model for a particular task needs to be evaluated, this is absolutely mandatory. However, we stress that the test set needs 1) to be truly independent (preferably not existing at the training time, see, e.g., a recent competition on Alzheimer's disease prediction for a good example [34,35]), and 2) needs to model the actual task as well as possible (i.e, collecting the test set at a hospital A when the actual method is to be used at a different hospital B may not be optimal).…”