Heavy metals (HMs) entering soils are redistributed by natural processes such as surface runoff and soil erosion. These natural processes are characterized by topographic factors (TFs, e.g., Topographic Wetness Index (TWI), Total Catchment Area (TCA), Slope, and Aspect), which are commonly quantified by a digital elevation model (DEM) of a certain spatial resolution. Nevertheless, few studies have examined how DEM resolutions affect the detection of influential TFs of soil HMs. In this study, we first applied the GeoDetector method to explore the coupling between the eight TFs and the concentrations of eight soil HMs under a gradient of DEM resolutions. We found that the important sorting of eight TFs on one HM at different resolutions is inconsistent. For example, for Hg, TWI emerged to be the dominant factor among the eight TFs at 90 m resolution, whereas TCA took the lead at 3000 m resolution. Moreover, the results strongly deny the existence of an optimal resolution (OR) among the HMs for any specific topographical factor. We further applied a source apportionment model (Positive Matrix Factorization—PMF) to explore the effects of five identified pollution sources and the underlying environmental processes on the inconsistent ORs. The main reason for such OR inconsistency is that each HM may be released from various sources and subsequently undergo environmental processes that are topographically modulated at different spatial scales. The main reason for such OR inconsistency is that each HM may have various sources and subsequent environmental processes that happen at different spatial scales. Moreover, each TF could simultaneously reflect different transport and transformation processes. Therefore, the apparent OR for one metal is jointly composed of the preferences of all the sources it contains; thus, it cannot be determined by the OR preferences of a single source alone. Based on the composition and intensity of pollution sources, we propose three possible strategies for a more robust GeoDetector analysis. The findings reported here provide new insights into the proper use of GeoDetector for selecting the appropriate DEM resolutions when identifying influential environmental factors.