Human settlements are areas closely related to human production and life, and their quality directly affects people’s physical health and quality of life. However, with air pollution continuing to worsen, people are becoming increasingly concerned about the exposure to air pollution in their residential regions. In addition, many studies ignore the long-term impact of environmental factors on the evolution of human settlement quality (HSQ). This study aims to assess the HSQ in the Mid-Spine Belt of Beautiful China (MSBBC) from 2000 to 2020 and to analyze the changes in its driving forces during different time periods. We divided the HSQ into five dimensions: terrain conditions, climate conditions, hydrological conditions, ground cover conditions, and air pollution exposure. The Entropy-TOPSIS and grey relational degree model were used to assess the HSQ in the MSBBC. To analyze the driving forces of HSQ, the optimal-parameters-based geographic detector model was utilized. The results show the following: (1) Within the study area, the degree of the population exposure risk to PM2.5 and its change trend were significantly different on both sides of the Hu Line, with the east showing higher levels than the west. (2) The HSQ of the MSBBC decreased from east to west. Moreover, the HSQ in the metropolitan area of the urban agglomeration was characterized from low to high. The HSQ underwent three phases, consisting of an increase (2000–2010), a decrease (2010–2015), and an increase again (2015–2020). (3) Among the driving factors, the impact of PM2.5 on the HSQ weakened year by year. The driving force of meteorological parameters on the HSQ was the strongest. Furthermore, the interactions of each factor could enhance the HSQ. The results of this study not only provide a strong reference for creating livable areas and promoting sustainability in the MSBBC but also contribute to addressing challenges such as pollution and climate change.