Background: Understanding the levels of health literacy among different groups is essential for better public health interventions targeting specific subgroups of the population. Additionally, this article explores the prevalence and influencing factors of the health literacy levels of different age groups during the COVID-19 epidemic.Methods: Multistage stratified cluster random sampling and the Probability Proportion to Size (PPS) method were used to select permanent residents aged 15–69 in Chongqing (54,706) for the questionnaire survey. The survey period is from July 2019 and July 2020. Single-factor analysis and logistic regression models were used to study the relationship between demographics, socioeconomic factors, other independent covariates, and health literacy.Results: The health literacy levels of residents declined with age, and there were significant differences in health literacy levels between age groups (χ2 = 3332.884, P < 0.05). As far as the factors affecting health literacy level are concerned, high education and high income are the protective factors for health literacy level for residents of all ages. For adolescents (OR = 1.383, 95% CI: 1.217–1.571), young adults (OR = 1.232, 95% CI = 1.117–1.358), and middle-aged people (OR = 1.096, 95% CI = 1.017–1.182), residence in rural areas was a protective factor. In terms of the dimensions of health literacy, in particular, elderly health literacy in 2020 in Scientific Health Concepts, Safety and First Aid, Basic Medical Care decreased significantly compared with 2019.Conclusions: For adolescents, young adults, middle-aged people, to solve the problem of urban and rural health quality gap, we should not only use the geographical division, but also consider the social population and socio-economic differences. For the elderly, the following four dimensions of health literacy need to be paid more attention than those of other age: Basic Knowledge and Concepts, Scientific Health Concepts, Safety and First Aid, and Basic Medical Care. A lack of knowledge on the prevention and treatment of chronic diseases is the main reason for the recent decline in health literacy. And the health literacy among residents in major public health emergencies is needed.