Animal models of Alzheimer’s disease (AD) have suggested that tau pathology propagation, facilitated by amyloid pathology, may occur along connected pathways. To investigate these ideas in humans, we combined amyloid scans with longitudinal data on white matter connectivity, hippocampal volume, tau positron emission tomography and memory performance in 256 cognitively healthy older individuals. Lower baseline hippocampal volume was associated with increased mean diffusivity of the connecting hippocampal cingulum bundle (HCB). HCB diffusivity predicted tau accumulation in the downstream-connected posterior cingulate cortex (PCC) in amyloid positive, not in amyloid negative individuals. Furthermore, HCB diffusivity predicted memory decline in amyloid positive individuals with high PCC tau binding. Our results provide in vivo evidence that higher amyloid pathology strengthens the association between HCB diffusivity and tau accumulation in the down-stream PCC and, facilitates memory decline. This confirms amyloid’s crucial role in potentiating neural vulnerability and cognitive decline marking the onset of preclinical AD.