Rapid urbanization greatly alters land surface vegetation cover and heat distribution, leading to the development of the urban heat island (UHI) effect and seriously affecting the healthy development of cities and the comfort of living. As an indicator of urban health and livability, monitoring the distribution of land surface temperature (LST) and discovering its main impacting factors are receiving increasing attention in the effort to develop cities more sustainably. In this study, we analyzed the spatial distribution patterns of LST of the city of Wuhan, China, from 2013 to 2019. We detected hot and cold poles in four seasons through clustering and outlier analysis (based on Anselin local Moran’s I) of LST. Furthermore, we introduced the geographical detector model to quantify the impact of six physical and socio-economic factors, including the digital elevation model (DEM), index-based built-up index (IBI), modified normalized difference water index (MNDWI), normalized difference vegetation index (NDVI), population, and Gross Domestic Product (GDP) on the LST distribution of Wuhan. Finally, to identify the influence of land cover on temperature, the LST of croplands, woodlands, grasslands, and built-up areas was analyzed. The results showed that low temperatures are mainly distributed over water and woodland areas, followed by grasslands; high temperatures are mainly concentrated over built-up areas. The maximum temperature difference between land covers occurs in spring and summer, while this difference can be ignored in winter. MNDWI, IBI, and NDVI are the key driving factors of the thermal values change in Wuhan, especially of their interaction. We found that the temperature of water area and urban green space (woodlands and grasslands) tends to be 5.4 °C and 2.6 °C lower than that of built-up areas. Our research results can contribute to the urban planning and urban greening of Wuhan and promote the healthy and sustainable development of the city.