(1) Background: The COVID-19 pandemic has not only changed people’s health behavior, but also induced a psychological reaction among the public. Research data is needed to develop scientific evidence-driven strategies to reduce adverse mental health effects. The aims of this study are to evaluate the anxiety reaction of Chinese people and the related determinants during the earliest phase of the COVID-19 outbreak in China. Evidence from this survey will contribute to a targeted reference on how to deliver psychological counseling service in the face of outbreaks. (2) Methods: A cross-sectional, population-based online survey was conducted from 28 January to 5 February 2020 using an open online questionnaire for people aged 18 years or above, residing in China and abroad. The socio-demographic information of the respondents was collected, and anxiety scores were calculated. A direct standardization method was used to standardize anxiety scores and a general linear model was used to identify associations between some factors (e.g., sex, age, education, etc.) and anxiety scores. (3) Results: A total of 10,946 eligible participants were recruited in this study, with a completion rate of 98.16% (10,946/11,151). The average anxiety score was 6.46 ± 4.12 (total score = 15); women (6.86 ± 4.11) scored higher than men (5.67 ± 4.04). The age variable was inversely and significantly associated with the anxiety score (β = −2.12, 95% CI: −2.47–−1.78). People possessing higher education (β = 1.15, 95% CI: 0.88–1.41) or a higher awareness of cognitive risk (β = 4.89, 95% CI: 4.33–5.46) reported higher levels of anxiety. There was a close association between poor subjective health and anxiety status (β = 2.83, 95% CI: 2.58–3.09). With the increase of confidence, the anxiety of the population exhibited a gradual decline (β = −2.45, 95% CI: −2.77–−2.13). (4) Conclusion: Most people were vulnerable to anxiety during the earliest phase of the COVID-19 outbreak in China. Younger women, individuals with high education, people with high cognitive risk and subjective poor health were vulnerable to anxiety during the epidemic. In addition, increasing confidence in resisting this pandemic is a protective determinant for individuals to develop anxiety. The findings suggest that policymakers adopt psychosocial interventions to reduce anxiety during the pandemic.