Thinner retinal nerve fiber layer (RNFL) has been reported in Alzheimer’s disease (AD) patient. However, whether changes in RNFL thickness can predict the cognitive deterioration remains unknown. We therefore set out a prospective clinical investigation to determine the potential association between the attenuation of RNFL thickness and the deterioration of cognitive function over a period of 25 months. We assessed cognitive function using the Repeatable Battery for the Assessment of Neuropsychological Status and measured RNFL thickness employing optical coherence tomography in 78 participants (mean age 72.31 ± 3.98 years, 52% men). The participants were categorized as stable participants whose cognitive status remained no change (N = 60) and converted participants whose cognitive status deteriorated (N = 18). We found that there was an association between the attenuation of superior quadrant RNFL thickness and the deterioration of cognitive function in the stable participants. In the converted participants, however, there was an inverse association between the reduction of inferior quadrant RNFL thickness and decline of cognitive functions [scores of list recall (R = -0.670, P = 0.002), adjusted (R = -0.493, P = 0.031)]. These data showed that less reduction in the inferior quadrant of RNFL thickness might indicate a higher risk for the patients to develop cognitive deterioration. These findings have established a system to embark a larger scale study to further test whether changes in RNFL thickness can serve as a biomarker of AD, and would lead to mechanistic studies to determine the cellular mechanisms of cognitive deterioration.