African Americans have double the prevalence of Alzheimer’s disease (AD), as compared to European Americans. However, the underlying causes of this health disparity are due to a multitude of environmental, lifestyle, and genetic factors that are not yet fully understood. Here, we review the effects of the two largest genetic risk factors for AD in African Americans: Apolipoprotein E (APOE) and ABCA7. We will describe the direct effects of genetic variation on neural correlates of cognitive function and report the indirect modulating effects of genetic variation on modifiable AD risk factors, such as aerobic fitness. As a means of integrating previous findings, we present a novel schematic diagram to illustrate the many factors that contribute to AD risk and impaired cognitive function in older African Americans. Finally, we discuss areas that require further inquiry, and stress the importance of racially diverse and representative study populations.