Age-related cognitive decline is linked to numerous molecular, structural, and functional changes in the brain. However, physical activity is a promising method of reducing unfavorable age-related changes. Physical activity exerts its effects on the brain through many molecular pathways, some of which are regulated by genetic variants in humans. In this paper, we highlight genes including apolipoprotein E (APOE), brain derived neurotrophic factor (BDNF), and catechol-O-methyltransferase (COMT) along with dietary omega-3 fatty acid, docosahexaenoic acid (DHA), as potential moderators of the effect of physical activity on brain health. There are a growing number of studies indicating that physical activity might mitigate the genetic risks for disease and brain dysfunction and that the combination of greater amounts of DHA intake with physical activity might promote better brain function than either treatment alone. Understanding whether genes or other lifestyles moderate the effects of physical activity on neurocognitive health is necessary for delineating the pathways by which brain health can be enhanced and for grasping the individual variation in the effectiveness of physical activity interventions on the brain and cognition. There is a need for future research to continue to assess the factors that moderate the effects of physical activity on neurocognitive function.