The APOE gene alleles modify human aging and the response to the diet at many levels with diverse pleotropic effects from gut to brain. To understand the interactions of APOE isoforms and diet, we analyze how cellular trafficking of apoE proteins affects energy metabolism, the immune system, and reproduction. The age-accelerating APOE4 allele alters the endosomal trafficking of cell surface receptors that mediate lipid and glucose metabolism. The APOE4 allele is the ancestral human allele, joined by APOE3 and then APOE2 in the human species. Under conditions of high infection, uncertain food, and shorter life expectancy, APOE4 may be adaptive for reducing mortality. As humans transitioned into modern less-infectious environments and longer life spans, APOE4 increased risks of aging-related diseases, particularly impacting arteries and the brain. The association of APOE4 with glucose dysregulation and body weight promotes many aging-associated diseases. Additionally, the APOE gene locus interacts with adjacent genes on chromosome 19 in haplotypes that modify neurodegeneration and metabolism, for which we anticipate complex gene-environment interactions. We summarize how diet and Alzheimer's disease (AD) risk are altered by APOE genotype in both animal and human studies and identify gaps. Much remains obscure in how APOE alleles modify nutritional factors in human aging. Identifying risk variant haplotypes in the APOE gene complex will clarify homeostatic adaptive responses to environmental conditions.