From ‘loose fitting’ to high-performance, uncertainty-aware brain-age modelling

Mind the gap: Performance metric evaluation in brain‐age prediction

“…Brain‐predicted age estimates are also promising in terms of predicting prognosis in diseases such as dementia (Biondo et al, 2020 ; Gaser et al, 2013 ; Wang et al, 2019 ) and multiple sclerosis (Cole et al, 2020 ; Høgestøl et al, 2019 ). From a methodological point of view, prediction models can benefit from advancements such as incorporating uncertainties into the predictions (Hahn et al, 2021 ; Marquand et al, 2019 ; Peng, Gong, Beckmann, Vedaldi, & Smith, 2021 ). Predicted age estimates are currently represented by a single value per individual, and while MAE and RMSE values describe overall model errors, an uncertainty measure per estimate could provide a realistic accuracy range for each individual's brain‐predicted age.…”

“…Although not a main focus in the current study, an increasingly common scenario involves combining data from various cohorts and scanners, which poses additional challenges related to site‐ and scanner‐dependent variance (Alfaro‐Almagro et al, 2021 ; Solanes et al, 2021 ; Tønnesen et al, 2020 ). Improving methods for site/scanner adjustments (Bayer et al, 2021 ; Dinga, Schmaal, Penninx, Veltman, & Marquand, 2020 ), or incorporating uncertainties into the predictions (Hahn et al, 2021 ; Marquand et al, 2019 ), represent promising avenues for further developing robust and valid biomarkers for brain health and disease. As evident from the current results, clear reporting of sample characteristics and model attributes is important to enable accurate interpretation of model performance metrics in future work.…”

“…Brain-predicted age estimates are also promising in terms of predicting prognosis in diseases such as dementia (Biondo et al, 2020;Gaser et al, 2013;Wang et al, 2019) and multiple sclerosis (Cole et al, 2020;Høgestøl et al, 2019). From a methodological point of view, prediction models can benefit from advancements such as incorporating uncertainties into the predictions (Hahn et al, 2021;Marquand et al, age. This could be obtained by using bootstrapping (Efron & Tibshirani, 1994): N (e.g., 500) different versions of the training set are created using random sampling with replacement.…”

Editorial: Predicting Chronological Age From Structural Neuroimaging: The Predictive Analytics Competition 2019

“…Conceptually, as proposed in Bashyam et al ( 15 ), with increased performance of brain age models, the possibility of models learning to completely correct for altered brain aging (e.g., caused by disease or life style) by extrapolation from the training data, could counter the utility of the brain-age as a biomarker. While ( 16 ) showed that this effect has not yet been empirically observed, the argument points toward a conceptual issue which could arise if models improve so much that they implicitly correct brain age for diseases. While certainly true, this issue could prove to be invaluable as such models would—by definition—enable an accurate direct classification of patients.…”

Deep neural networks learn general and clinically relevant representations of the ageing brain

“…There is an ongoing discussion in the field on whether brain age models that are precise, or those that allow for sufficient variance in their single-subject predictions, are the most useful in a downstream analysis of behavioural and clinical traits 32,55,56 . An argument for a model which allows more variation (a looser fit) is that this would more accurately depict brain age as a complex process which appears differently in different individuals.…”