PM2.5 particles with an aerodynamic diameter of less than 2.5 μm are receiving increasing attention in China. Understanding how complex factors affect PM2.5 particles is crucial for the prevention of air pollution. This study investigated the influence of meteorological factors and land use on the dynamics of PM2.5 concentrations in four urban agglomerations of China at different scales from 2010 to 2020, using the Durbin spatial domain model (SDM) at five different grid scales. The results showed that the average annual PM2.5 concentration in four core urban agglomerations in China generally had a downward trend, and the meteorological factors and land use types were closely related to the PM2.5 concentration. The impact of temperature on PM2.5 changed significantly with an increase in grid scale, while other factors did not lead to obvious changes. The direct and spillover effects of different factors on PM2.5 in inland and coastal urban agglomerations were not entirely consistent. The influence of wind speed on coastal urban clusters (the Pearl River urban agglomeration (PRD) and Yangtze River urban agglomeration (YRD)) was not significant among the meteorological factors, but it had a significant impact on inland urban clusters (the Beijing–Tianjin–Hebei urban agglomeration (BTH) and Chengdu–Chongqing urban agglomeration (CC)). The direct effect of land use type factors showed an obvious U-shaped change with an increase in the research scale in the YRD, and the direct effect of land use type factors was almost twice as large as the spillover effect. Among land use type factors, human factors (impermeable surfaces) were found to have a greater impact in inland urban agglomerations, while natural factors (forests) had a greater impact in coastal urban agglomerations. Therefore, targeted policies to alleviate PM2.5 should be formulated in inland and coastal urban agglomerations, combined with local climate measures such as artificial precipitation, and urban land planning should be carried out under the consideration of known impacts.