Oxytocin is deeply involved in human relations. In recent years, it is becoming clear that oxytocin is also involved in social cognition and social behavior. Oxytocin receptors are also thought to be present in the hippocampus and amygdala, and the relationship between oxytocin and the structure and function of the hippocampus and amygdala has been reported. However, few studies have investigated oxytocin and its relationship to hippocampus and amygdala volume in elderly people. The aim of this study is to investigate the association between serum oxytocin levels and hippocampus and amygdala volume in elderly people. The survey was conducted twice in Kurokawa-cho, Imari, Saga Prefecture, Japan, among people aged 65 and older. We collected data from 596 residents. Serum oxytocin level measurements, brain MRI, Mini-Mental State Examination and Clinical Dementia Rating were performed in Time 1 (2009–2011). Follow-up brain MRI, Mini-Mental State Examination and Clinical Dementia Rating were performed in Time 2 (2016–2017). The interval between Time 1 and Time 2 was about 7 years. Fifty-eight participants (14 men, mean age 72.36 ± 3.41 years, oxytocin 0.042 ± 0.052 ng/ml; 44 women, mean age 73.07 ± 4.38 years, oxytocin 0.123 ± 0.130 ng/ml) completed this study. We analyzed the correlation between serum oxytocin levels (Time 1) and brain volume (Time 1, Time 2, and Time 1–2 difference) using voxel-based morphometry implemented with Statistical Parametric Mapping. Analysis at the cluster level (family-wise error; P < 0.05) showed a positive correlation between serum oxytocin levels (Time 1) and brain volume of the region containing the left hippocampus and amygdala (Time 2). This result suggests that oxytocin in people aged 65 and older may be associated with aging-related changes in hippocampus and amygdala volume.