The prevalence of dementia increases with nearly 10 million new cases each year, with Alzheimer’s disease contributing to 60–70% of cases. Environmental factors such as drinking water have been evaluated to determine if a relationship exists between trace elements in drinking water and the risk of developing cognitive disorders in the elderly. The purpose of the current systematic review was to evaluate an association between the composition of drinking water and cognitive function in the elderly. In accordance with the Preferred Reporting Items for Systematic Review and Meta-Analyses guidelines, a literature search was conducted using PubMed and CINAHL databases. A total of 10 studies were included in the current systematic review. Aluminum is the most commonly evaluated trace element in studies (n = 8), followed by silica (n = 5), calcium (n = 4), and fluoride (n = 4). Aluminum exposure showed an increased risk of cognitive decline in four studies, with no association reported in the other studies. Higher silica and pH levels were shown to be protective against a decline in cognitive function. A similar protective effect of calcium was found in two studies. Future research should measure multiple trace mineral levels in all water sources to evaluate the impact on cognitive function.