Progression Models for Imaging Data with Longitudinal Variational Auto Encoders

Abstract

Disease progression models are crucial to understanding degenerative diseases. Mixed-effects models have been consistently used to model clinical assessments or biomarkers extracted from medical images, allowing missing data imputation and prediction at any timepoint. However, such progression models have seldom been used for entire medical images. In this work, a Variational Auto Encoder is coupled with a temporal linear mixed-effect model to learn a latent representation of the data such that individual trajectories follow straight lines over time and are characterised by a few interpretable parameters. A Monte Carlo estimator is devised to iteratively optimize the networks and the statistical model. We apply this method on a synthetic data set to illustrate the disentanglement between time dependant changes and inter-subjects variability, as well as the predictive capabilities of the method. We then apply it to 3D MRI and FDG-PET data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) to recover well documented patterns of structural and metabolic alterations of the brain.

Generative AI models in time varying biomedical data: a systematic review (Preprint)

“…In dealing with multi-modality inputs, a combination of deep learning models dealing with each modality may also be advantageous. This work combined VAE with a linear mixed-effect model to learn a Riemman progression model that can be applied to general disease trajectory estimation [118]. Another work combined an autoencoder framework with attention units in a transformer to predict final ischemic stroke lesions from MRI [119].…”

Prediction of white matter hyperintensities evolution one-year post-stroke from a single-point brain MRI and stroke lesions information

“…These approaches model and predict spatial changes of specific disease features such as evolution of WMH, enlargement of ventricles, and brain atrophy. Other examples are predicting lung nodule progression of pulmonary tumour (Rafael-Palou et al, 2022), predicting dynamic change of brain structures from healthy individuals to MCI and AD patients (Sauty and Durrleman, 2022), and studies for predicting the evolution of WMH in brain images of stroke patients (Rachmadi et al, 2019(Rachmadi et al, , 2020(Rachmadi et al, , 2021 1 https://github.com/febrianrachmadi/probunet-gan-vie…”

“…These approaches model and predict spatial changes of specific disease features such as evolution of WMH, enlargement of ventricles, and brain atrophy. Other examples are predicting lung nodule progression of pulmonary tumour (Rafael-Palou et al, 2022), predicting dynamic change of brain structures from healthy individuals to MCI and AD patients (Sauty and Durrleman, 2022), and studies for predicting the evolution of WMH in brain images of stroke patients (Rachmadi et al, 2019(Rachmadi et al, , 2020(Rachmadi et al, , 2021 The present study belongs to the third category, in which a predictive model is used to estimate spatial dynamic changes of the evolution of WMH identified on an MRI scan. This third category is the most challenging because of the complexity and resolution of the data/image being predicted.…”

“…Other examples are predicting lung nodule progression of pulmonary tumour 36 , predicting dynamic change of brain structures from healthy individuals to MCI and AD patients 37 , and studies for predicting the evolution of WMH in brain images of stroke patients [23][24][25] This study belongs to the third category, in which a predictive model is used to spatially estimate the dynamic changes of WMH on an MRI scan at a certain time point. This third category is the most challenging because of the complexity and resolution of the data/image being predicted, especially when the time-point estimated is close to the baseline scan.…”

Deep learning application on head CT images

“…However, it is difficult to obtain these paired images as a subject cannot be healthy and unhealthy at the same time given their results. The alternative is to collect the nearest approximation from longitudinal data [86], which may be a suboptimal solution since many of the available public medical datasets contain only independent images. Data generation techniques such as score matching modes, likelihood based models and adversarial generative models [87][88][89][90] have had a huge success in generating images or learning better semantic segmentation.…”