The influence of different urbanization levels on land surface temperature (LST) has attracted extensive attention. Though both are world megacities, Shanghai and Tokyo have gone through different urbanization processes that urban sprawl characterized by impervious surfaces was more notable in Shanghai than in Tokyo over the past years. Here, annual and seasonal mean LST in daytime (LSTday), in nighttime (LSTnight), and LSTdiff (annual and seasonal mean difference of LST in daytime and nighttime) were extracted from the MODIS LST product, MYD11A2 006, for 9 typical sites in Shanghai and Tokyo from 2003 to 2018, respectively. Then the effects of the urbanization levels were analyzed through Mann-Kendall statistics and Sen’s slope estimator. The trends of change in LSTday and LSTdiff for most sites in Shanghai, an urbanizing region, rose. In addition, there was no obvious regularity when considering seasonal factors, which could be due to the increasing fragmentized landscapes and scattered water bodies produced by urbanization. By comparison, the change in LST in Tokyo, a post-urbanizing region, was regular, especially in the spring. In other seasons, there was no obvious trend in temperature change regardless of whether the land cover was impervious surface or mountain forest. On the whole, vegetation cover and water bodies can mitigate the urban heat island (UHI) effect in urban regions. For more scientific urban planning, further analysis about the effect of urbanization on LST should focus on the compound stress from climate change and urbanization.