In the process of China’s rapid urbanization, the health level of residents has been improved to a great extent. However, with the expansion of urban scale and spatial restructuring, a series of urban environmental problems have posed new challenges to public health. However, the impact of the built environment on residents’ health is controversial, and the applicability of the conclusions based on western urban sprawl in China is not clear enough. In addition, the exploration of the impact path of the built environment on health is still not comprehensive and in-depth. Based on the China Labor Dynamics Survey (CLDS) in 2016 and relevant statistical yearbook data, this study explored the impact of the built environment at community and urban scale on residents’ health and its age heterogeneity and further explored the mediating role of physical exercise, neighborhood support, and community safety. According to the research, the urban and community-built environment has significant impacts on residents’ health, and the impact is significantly different at different scales. In addition, there is a significant difference in the impact of built environment factors on residents’ health among populations with different life cycles. From the perspective of the impact path, greening coverage can improve residents’ self-rated health by enhancing the perceived safety of living in the community. In contrast, the high community population density will not only weaken the degree of neighborhood support but also reduce the perception level of community residential safety, thus damaging residents’ health. In short, from the perspective of environmental intervention, the previously mentioned results put forward possible suggestions on strengthening the construction of a healthy living environment so as to maximize the health effectiveness of cities and communities.