It is now acknowledged that Alzheimers Disease (AD) processes are present decades before the onset of clinical symptoms, but it remains unknown whether lifestyle factors can protect against these early AD processes in midlife. We asked whether modifiable lifestyle activities impact cognition in middle aged individuals who are cognitively healthy, but at risk for late life AD. Participants (40 to 59 years) completed cognitive and clinical assessments at baseline (N = 210) and two years follow up (N = 188). Mid-life activities were measured with the Lifetime of Experiences Questionnaire. We assessed the impact of lifestyle activities, known risk factors for sporadic late onset AD (Apolipoprotein E Ɛ4 allele status, family history of dementia, and the Cardiovascular Risk Factors Aging and Dementia score), and their interactions on cognition. More frequent engagement in physically, socially and intellectually stimulating activities was associated with better cognition (verbal, spatial and relational memory), at baseline and follow up. Critically, more frequent engagement in these activities was associated with stronger cognition (verbal and visuospatial functions, and conjunctive short-term memory binding) in individuals with family history of dementia. Impaired visuospatial function is one of the earliest cognitive deficits in AD and has previously associated with increased AD risk in this cohort. Additionally, conjunctive memory functions have been found impaired in the presymptomatic stages of AD. These findings suggest that modifiable lifestyle activities offset cognitive decrements due to AD risk in midlife and support the targeting of modifiable lifestyle activities for the prevention of Alzheimers Disease.