With age, eyesight declines and the vulnerability to age-related eye diseases such as glaucoma, cataract, macular degeneration and diabetic retinopathy increases. With the aging of the global population, the prevalence of these diseases is projected to increase, leading to reduced quality of life and increased healthcare cost. In the following, we built an eye age predictor by training convolutional neural networks to predict age from 175,000 eye fundus and optical coherence tomography images (R-Squared=83.6+/-0.6%; root mean squared error=3.34+/-0.07 years). We used attention maps to identify the features driving the eye age prediction. We defined accelerated eye aging as the difference between eye age and chronological age and performed a genome wide association study [GWAS] on this phenotype. Accelerated eye aging is 28.2+-1.2% GWAS-heritable, and is associated with 255 single nucleotide polymorphisms in 122 genes (e.g HERC2, associated with eye pigmentation). Similarly, we identified biomarkers (e.g blood pressure), clinical phenotypes (e.g chest pain), diseases (e.g cataract), environmental variables (e.g sleep deprivation) and socioeconomic variables (e.g income) associated with our newly defined phenotype. Our predictor could be used to detect premature eye aging in patients, and to evaluate the effect of emerging rejuvenation therapies on eye health.