Globally, vaccine hesitancy is a growing public health problem. It is detrimental to the consolidation of immunization program achievements and elimination of vaccine-targeted diseases. The objective of this study was to estimate the prevalence of COVID-19 vaccine hesitancy in China and explore its contributing factors. A national cross-sectional online survey among Chinese adults (≥18 years old) was conducted between August 6, 2021 and August 9 via a market research company. We collected sociodemographic information; lifestyle behavior; quality of life; the knowledge, awareness, and behavior of COVID-19; the knowledge, awareness, and behavior of COVID-19 vaccine; willingness of COVID-19 vaccination; accessibility of COVID-19 vaccination services; skepticism about COVID-19 and COVID-19 vaccine; doctor and vaccine developer scale; and so on. Odds ratios (OR) with 95% confidence intervals (CI) were used to estimate the associations by using logistic regression models. A total of 29,925 residents (48.64% men) were enrolled in our study with mean age of 30.99 years. We found an overall prevalence of COVID-19 vaccine hesitancy at 8.40% (95% CI, 8.09–8.72) in primary vaccination and 8.39% (95% CI, 8.07–8.70) in booster vaccination. In addition, after adjusting for potential confounders, we found that women, higher educational level, married residents, higher score of health condition, never smoked, increased washing hands, increased wearing mask, increased social distance, lower level of vaccine conspiracy beliefs, disease risks outweigh vaccine risk, higher level of convenient vaccination, and higher level of trust in doctor and developer were more willing to vaccinate than all others (all p < 0.05). Age, sex, educational level, marital status, chronic disease condition, smoking, healthy behaviors, the curability of COVID-19, the channel of accessing information of COVID-19 vaccine, endorsement of vaccine conspiracy beliefs, weigh risks of vaccination against risks of the disease, making a positive influence on the health of others around you, and lower trust in healthcare system may affect the variation of willingness to take a COVID-19 vaccine (all p < 0.05). The prevalence of COVID-19 vaccine hesitancy was modest in China, even with the slight resulting cascade of changing vaccination rates between the primary and booster vaccination. Urgent action to address vaccine hesitancy is needed in building trust in medical personnel and vaccine producers, promoting the convenience of vaccination services, and spreading reliable information of COVID-19 vaccination via the Internet and other media.