The integrity and resilience of our environment are confronted with unprecedented challenges, stemming from the escalating pressures of urban expansion and the need for ecological preservation. This study proposes an Improved Remote Sensing Ecological Index (IRSEI), which employs humidity (WET), the Normalized Difference Vegetation Index (NDVI), Land Surface Temperature (LST), a standardized Building–Bare Soil Index (NDBSI), aerosol optical depth (AOD), and the comprehensive salinity index (CSI). The IRSEI model was utilized to assess the ecological quality of Hangzhou over the period from 2003 to 2023. Additionally, the random forest model was employed to analyze the factors driving ecological quality. Furthermore, the gradient effect in the horizontal direction away from the urban center was examined using the buffer zone method. Our analysis reveals the following: (1) approximately 95% of the alterations in ecological quality observed from 2003 to 2023 exhibited marginal improvements, declines, or were negligible; (2) the transformations in IRSEI during this period, including variations in surface temperature and transportation networks, exhibited strong correlations (0.85) with human activities. Moreover, the influence of AOD and the comprehensive salinity index on IRSEI demonstrated distinct spatial disparities; (3) the IRSEI remained generally stable up to 30 km outside the city center, indicating a trend of agglomeration in the center and significant areas in the surroundings. The IRSEI serves as a robust framework for bolstering the assessment of regional ecological health, facilitating ecological preservation and rejuvenation efforts, and fostering coordinated sustainable regional development.