Intervention studies on sedentary older adults have demonstrated that commencing physical exercise at an older age has a positive effect on brain structure. Although this suggests that older athletes with lifelong sports training have larger gray matter volume (GMV) in some brain regions compared to age-matched non-athletes, evidence in the literature is scarce. Moreover, it remains unclear whether a larger GMV is associated with training intensity or period of training in life. To address these gaps in the literature, we compared regional brain GMV between 24 older athletes (mean age, 71.4 years; age at the commencement of sports training, 31.2 years, continuous sports training, 40.0 years; current training time, 7.9 h/week) and 24 age-matched non-athletes (mean age, 71.0 years). The period of sports training and the current training time of the athletes were assessed. Both groups were evaluated for physical activity intensity as well as cognitive and motor performance. Although no group differences were noted in cognitive and motor performance, athletes reported higher physical activity intensity than non-athletes. Whole-brain structural analysis revealed a significantly larger GMV in several brain regions in athletes. Notably, the GMV of the precuneus in athletes was positively correlated with earlier commencement of sports training and training duration but was negatively correlated with current training time. Our findings demonstrate that early-commenced and continued sports training predicts structural maintenance of the precuneus in old age. Our results also suggest that excessive training time in old age may have a negative impact on the GMV of the precuneus; thereby delineating how the precuneus is associated with lifelong sports training in older athletes.