Green development is the core goal for contemporary urban areas. It has become essential to explore new types of urban green development, and the concept of the “national civilized city” which is the most influential city brand in China, has emerged. Drawing upon the administrative competing theory, based on the panel data of 281 cities in China from 2000 to 2018, this paper employs a propensity scores matching (PSM) design and a difference-in-difference (DID) approach to examine the influence of the establishment of the national civilized city policy on the green development of prefecture-level cities in China. First, the result shows that the establishment of the national civilized city policy can significantly improve the urban green total factor productivity (GTFP). Second, the mediation mechanism analyses show that the establishment of national civilized city policy can promote cities to increase their investment in R&D, increase the financial investment in environmental governance, and optimize the industrial structure, which further lays a solid foundation for urban green development. Third, the heterogeneity analysis shows that the impact of civilized city construction on urban GTFP is different in terms of population and economic scales. The results show that the weight of environmental management and R&D investment should be increased in the evaluation index of the national civilized city, and the promotion of urban green development should be maximized.