Visual hallucinations carry poor prognosis in Parkinson’s disease. Here we tested the hypothesis that the hippocampus and visuospatial memory impairment play a central role in the pathology of PD with visual hallucinations. Multimodal magnetic resonance imaging of the brain was carried out in 12 people with PD and visual hallucinations; 15 PD individuals without hallucinations; and 14 healthy controls. Age, gender, cognitive ability, and education level were matched across the three groups. PD patients were taking dopaminergic medication. Hippocampal volume, shape, mean diffusivity (MD), and functional connectivity within the whole brain were examined. Visuospatial memory was compared between groups, and correlations with hippocampal MD, functional connectivity, and the severity of hallucinations were explored. There were no macrostructural differences across groups, but individuals with hallucinations had higher diffusivity in posterior hippocampus than the other two groups. Visuospatial memory was poorer in both PD groups compared to controls, and was correlated with hallucinations. Finally, hippocampal functional connectivity in the visual cortices was lower in those with hallucinations than other groups, and this correlated with visuospatial memory impairment. In contrast, functional connectivity between the hippocampus and default mode network regions and frontal regions was greater in the PD hallucinators compared to other groups. We suggest that hippocampal pathology, which disrupts visuospatial memory, makes a key contribution to visual hallucinations in PD. These findings may pave the way for future studies of imaging biomarkers to measure treatment response in those with PD who are most at risk of poor outcomes.Electronic supplementary materialThe online version of this article (doi:10.1007/s00429-014-0907-5) contains supplementary material, which is available to authorized users.