transferGWAS: GWAS of images using deep transfer learning

Abstract

Motivation Medical images can provide rich information about diseases and their biology. However, investigating their association with genetic variation requires non-standard methods. We propose transferGWAS, a novel approach to perform genome-wide association studies directly on full medical images. First, we learn semantically meaningful representations of the images based on a transfer learning task, during which a deep neural network is trained on independent but similar data. Then, we perform genetic association tests with these representations. Results We validate the type I error rates and power of transferGWAS in simulation studies of synthetic images. Then we apply transferGWAS in a genome-wide association study of retinal fundus images from the UK Biobank. This first-of-a-kind GWAS of full imaging data yielded 60 genomic regions associated with retinal fundus images, of which 7 are novel candidate loci for eye-related traits and diseases. Supplementary information Our method is implemented in Python and available at https://github.com/mkirchler/transferGWAS/

Eye-brain connections revealed by multimodal retinal and brain imaging genetics in the UK Biobank

“…Many CNNs models have been trained on ImageNet and were widely used in the image processing field to learn complex patterns from images. In addition to the ResNet50 129 model used by the transferGWAS 31 , we implemented 10 more pre-trained CNN models, including the AlexNet 130 , Vgg16 131 , Vgg19 131 , GoogLeNet (Inception V1) 132 , Inception (V3) 132 , ResNet18 129 , ResNet34 129 , SqueezeNet 133 , MobileNet 134 , and ShuffleNet 135 . These pre-trained models are available on Pytorch 136 and represent different designs and architectures, such as layer depth, size of kernels, and hyperparameters.…”

“…Many CNNs models have been trained on ImageNet and were widely used in the image processing field to learn complex patterns from images. In addition to the ResNet50 129 model used by the transferGWAS 31 , we implemented 10 more pre-trained CNN models, including the AlexNet 130 , Vgg16 131 , Vgg19 131 , GoogLeNet (Inception V1) 132 , Inception (V3) 132 We began by combining the original left and right retinal fundus images and the rotated images with 90°, 180°, and 270°, each with and without horizontal mirroring. Next, we input these eight retinal fundus images into each pre-trained model and averaged the outputs from the last layer of convolutional networks.…”

“…In addition, magnetic resonance imaging (MRI) captures both structural and functional characteristics of the brain, resulting in a wide range of clinical applications in neurological and neuropsychiatric disorders 24 . Recent large-scale genome-wide association studies (GWAS) have shown that both retinal imaging biomarkers [25][26][27][28][29][30][31][32][33] and brain MRI traits [34][35][36][37][38][39][40][41] are heritable, with the genetic influences of common genetic variants being identified in hundreds of genomic regions. As expected, genetic overlaps were identified between retinal imaging traits and eye disorders (such as the cupping of the optic nerve head and glaucoma 30 ) as well as between brain MRI traits and brain disorders (such as functional connectivity of the visual network and Alzheimer's disease 39 ).…”

“…The 46 OCT derived measurements were already available in the UKB database, such as retinal thickness across layers 43, 44 and vertical cup-to-disc ratio 29 . For the fundus images, we used 11 different pre-trained transfer learning 31 models built from the ImageNet 45 database to extract imaging features of the retinal structure. In each deep transfer learning model, the top 10 principal components (PCs) were considered, explaining an average of 70.71% variance (range = [50.58%, 95.84%]) in the last layer, resulting in 110 fundus image features (11 x 10).…”

“…In each deep transfer learning model, the top 10 principal components (PCs) were considered, explaining an average of 70.71% variance (range = [50.58%, 95.84%]) in the last layer, resulting in 110 fundus image features (11 x 10). These deep-learning-based image embeddings and low-dimensional representations contain eye-specific biological information, which may not be present in standard eye measurements 31 . We conducted GWAS for these 156 (46 + 110) retinal imaging traits and then evaluated their genetic connections with 458 imaging traits from three primary brain MRI modalities, including 1) 101 regional brain volumes 35 and 63 cortical thickness traits 46 from structural MRI; 2) 110 diffusion tensor imaging (DTI) parameters from diffusion MRI 37 ; and 3) 92 functional connectivity and activity (or amplitude) traits from resting state and task-based functional MRI (fMRI) 39 , respectively.…”