From YouTube to the brain: Transfer learning can improve brain-imaging predictions with deep learning

Malik, Nahiyan; Bzdok, Danilo

doi:10.1016/j.neunet.2022.06.014

Cited by 14 publications

(3 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These tasks presumably may also benefit from the pre-training methods examined here and are a topic for future work. Finally, very recent work has pretrained MRI deep learning methods on YouTube videos, 31 raising the intriguing possibility that natural image or video pretraining of MRI deep learning models may perform as well, in some cases, as pretraining on tasks in the medical domain.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Evaluation of transfer learning methods for detecting Alzheimer’s disease with brain MRI

Dhinagar

Thomopoulos²,

Rajagopalan³

et al. 2023

18th International Symposium on Medical Information Processing and Analysis

View full text Add to dashboard Cite

Deep neural networks show great promise for classifying brain diseases and making prognostic assessments based on neuroimaging data, but large, labeled training datasets are often required to achieve high predictive accuracy. Here we evaluated a range of transfer learning or pre-training strategies to create useful MRI representations for downstream tasks that lack large amounts of training data, such as Alzheimer’s disease (AD) classification. To test our proposed pretraining strategies, we analyzed 4,098 3D T1-weighted brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort and independently validated with an out-of-distribution test set of 600 scans from the Open Access Series of Imaging Studies (OASIS3) cohort for detecting AD. First, we trained 3D and 2D convolutional neural network (CNN) architectures. We tested combinations of multiple pre-training strategies based on (1) supervised, (2) contrastive learning, and (3) self-supervised learning - using pre-training data within versus outside the MRI domain. In our experiments, the 3D CNN pre-trained with contrastive learning provided the best overall results - when fine-tuned on T1-weighted scans for AD classification - outperformed the baseline by 2.8% when trained with all of the training data from ADNI. We also show test performance as a function of the training dataset size and the chosen pre-training method. Transfer learning offered significant benefits in low data regimes, with a performance boost of 7.7%. When the pretrained model was used for AD classification, we were able to visualize an improved clustering of test subjects' diagnostic groups, as illustrated via a uniform manifold approximation (UMAP) projection of the high-dimensional model embedding space. Further, saliency maps indicate the additional activation regions in the brain scan using pretraining, that then maximally contributed towards the final prediction score.

show abstract

Section: Discussion and Future Workmentioning

confidence: 99%

Evaluation of transfer learning methods for detecting Alzheimer’s disease with brain MRI

Dhinagar

Thomopoulos²,

Rajagopalan³

et al. 2023

18th International Symposium on Medical Information Processing and Analysis

View full text Add to dashboard Cite

show abstract

“…These tasks presumably may also benefit from the pre-training methods examined here and are a topic for future work. Finally, very recent work has pre-trained MRI deep learning methods on YouTube videos, 31 raising the intriguing possibility that natural image or video pre-training of MRI deep learning models may perform as well, in some cases, as pre-training on tasks in the medical domain.…”

Section: Discussion and Future Workmentioning

confidence: 99%

Evaluation of Transfer Learning Methods for Detecting Alzheimer’s Disease with Brain MRI

Dhinagar¹,

Thomopoulos²,

Rajagopalan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Deep neural networks show great promise for classifying brain diseases and making prognostic assessments based on neuroimaging data, but large, labeled training datasets are often required to achieve high predictive accuracy. Here we evaluated a range of transfer learning or pre-training strategies to create useful MRI representations for downstream tasks that lack large amounts of training data, such as Alzheimer's disease (AD) classification. To test our models, we analyzed 4,098 3D T1-weighted brain MRI scans from the Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort and independently validated our proposed methods for detecting AD with an out-of-distribution test set of 600 scans from the Open Access Series of Imaging Studies (OASIS3) cohort. First, we trained 3D and 2D convolutional neural network (CNN) architectures. We tested combinations of multiple pre-training strategies based on (1) supervised, (2) contrastive learning, and (3) self-supervised learning - using pre-training data within versus outside the MRI domain. In our experiments, the 3D CNN pre-trained with contrastive learning provided the best overall results - when fine-tuned on T1-weighted scans for AD classification - outperformed the baseline by 2.8% when trained with all of the training data from ADNI. We also show test performance as a function of the training dataset size and the chosen pre-training method. Transfer learning offered significant benefits in low data regimes, with a performance boost of 7.7%. When the pre-trained model was used for AD classification, we were able to visualize an improved clustering of test subjects' diagnostic groups, as illustrated via a uniform manifold approximation (UMAP) projection of the high-dimensional model embedding space. Further, saliency maps indicate the additional activation regions in the brain scan using pre-training, that then maximally contributed towards the final prediction score.

show abstract

“…[ 29 ] used the ImageNet database [ 20 ], a large, public dataset containing naturalistic images from more than 1,000 classes, to pretrain a model and adapt it to classify tasks from 2-dimensional (2D) fMRI data. This database was also used in [ 31 ] for pretraining a 2D structural MRI classifier. In the same study, the Kinetics dataset [ 32 ] was also used to evaluate the transfer learning process with 3D images.…”

Section: Introductionmentioning

confidence: 99%

On the benefits of self-taught learning for brain decoding

2022

View full text Add to dashboard Cite

Context We study the benefits of using a large public neuroimaging database composed of functional magnetic resonance imaging (fMRI) statistic maps, in a self-taught learning framework, for improving brain decoding on new tasks. First, we leverage the NeuroVault database to train, on a selection of relevant statistic maps, a convolutional autoencoder to reconstruct these maps. Then, we use this trained encoder to initialize a supervised convolutional neural network to classify tasks or cognitive processes of unseen statistic maps from large collections of the NeuroVault database. Results We show that such a self-taught learning process always improves the performance of the classifiers, but the magnitude of the benefits strongly depends on the number of samples available both for pretraining and fine-tuning the models and on the complexity of the targeted downstream task. Conclusion The pretrained model improves the classification performance and displays more generalizable features, less sensitive to individual differences.

show abstract

From YouTube to the brain: Transfer learning can improve brain-imaging predictions with deep learning

Cited by 14 publications

References 65 publications

Evaluation of transfer learning methods for detecting Alzheimer’s disease with brain MRI

Evaluation of transfer learning methods for detecting Alzheimer’s disease with brain MRI

Evaluation of Transfer Learning Methods for Detecting Alzheimer’s Disease with Brain MRI

On the benefits of self-taught learning for brain decoding

Contact Info

Product

Resources

About