Memory deficits are common during aging, but little is known about the impact of environmental and genetic variables on memory. The genes SLC30A3 and SEP15 are, respectively, responsible for transporting zinc and selenium, micronutrients that are neuroprotective agents. The aim of this study was to investigate the effect of nutrigenetic interactions on the memory scores of volunteers more than 50 years old. For this cross-sectional study, 240 individuals were enrolled. Micronutrient dosage was determined using atomic absorption spectrophotometry. The SNPs rs5859, rs5854, and rs561104 in SEP15 and rs73924411 and rs11126936 in SLC30A3 were determined by real-time PCR. The evaluations of verbal and visual memory were performed using the Weschler Memory Scalerevised and the Rey's verbal learning test. A gene versus nutrient interaction was observed for SLC30A3 rs73924411 and zinc concentration. Carriers of the T allele had higher scores for short-term and long-term verbal memories than CC homozygotes only when zinc serum concentration was below the recommended level (p value for the interaction for short-term verbal memory = 0.011, p value for the interaction for long-term verbal memory = 0.039). For SEP15, C carriers of the rs5845 SNP allele had higher verbal learning memory scores than TT homozygotes (0.13 ± 1.13 vs. -1.10 ± 1.20, p = 0.034). Our results suggest the influence of genetic polymorphisms on memory score and identify gene versus nutrient interactions between zinc serum concentration and memory score.