Background: Given the prevalence of the omicron variant and decreased immunity provided by vaccines, it is imperative to enhance resistance to COVID-19 in the old population. We planned to explore the hesitancy rate toward the booster dose of the COVID-19 vaccine and the association between risk perception and the abovementioned rate among people aged 60 and older. Methods: This national cross-sectional study was conducted in mainland China from 25 May to 8 June 2022, targeting people who were 60 years old or above. Four dimensions were extracted from the Health Belief Model (HBM) to assess participants’ perceived risk levels, including perceived susceptibility, perceived severity, perceived barriers, and perceived benefit. An independent Chi-square test was used to compare the vaccine hesitancy rates among different groups stratified by characteristics. Univariable and multivariable logistic regression models were performed to explore the associations between risk perception and hesitancy rate. Results: Of 3321 participants, 17.2% (95% CI: 15.9–18.5%) were hesitant about booster shots of COVID-19 vaccines. Believing that they were ineligible for vaccination due to certain illnesses (38.3%), concern about vaccine safety (32.0%), believing the booster shots were unnecessary (33.1%), and their limitation on movements (28.0%) were the main reasons for vaccine hesitation. Adjusted by all the selected covariates, people with low perception level of susceptibility (aOR = 1.39, 95% CI: 1.00–1.92) and benefit (low: aOR = 3.31, 95% CI: 2.01–5.45; moderate: aOR = 2.23, 95% CI: 1.75–2.85) were less likely to receiving the booster dose, and the same results were found in people with higher perceived barriers (moderate: aOR = 2.67, 95% CI: 2.13–3.35; high: aOR = 2.04, 95% CI: 1.14–3.67). Our estimates were stable in all four models. Conclusions: In total, 17.2% of the people aged 60 years and older in China were hesitant about booster dose of COVID-19 vaccines, and it was closely associated with a lower level of perceived susceptibility and benefit, as well as a higher level of perceived barriers. Concerns about contraindications, vaccine safety, and limited movements were the main reasons for vaccine hesitancy. Targeted public health measure is a priority to improve the understanding of the elderly on their own susceptibility and vulnerability and clear the obstacles to vaccination.