Although soil quality can be highly altered by mining activities, there are few reports on soil pollution in mining cities. We systematically characterized the heavy metals (HMs) pollution, risks, sources, and influencing factors in the surrounding soils of Shuozhou. Specifically, 146 samples were collected, and the potential ecological risk index (RI) and the single-factor index were jointly used to understand the environmental risk of HMs. Meanwhile, correlation analysis was applied to find the influencing factors of HMs. The results of the soil pollution risk assessment in the entire area of Shuozhou were compared with those in the open-pit mine area. (1) The mean concentrations of Cr, As, Cd, Pb, and Hg in our study were found to be higher than the background value. The RI results indicated that most soil samples (82.88%) in Shuozhou had a low potential ecological risk. Compared with the Pingshuo open-pit mine (average RI value: 200.07), the potential ecological RI was lower. (2) The HM correlation indicated that Cr and As were associated with the parent rock, whereas Cd, together with Hg and Pb, were associated with anthropic activities. (3) There was no significant correlation between HM concentrations and farmland slope. Located in the Datong Basin, the terrain of Shuozhou is relatively flat and open and has little impact on the distribution of HMs. (4) Only Hg and Pb have a negative correlation with pH. This suggests that soil with a lower pH value may be beneficial to the accumulation of Hg and Pb in soil. (5) Among the eight industry types examined, the pollution capacity level of the leather, fur, feather, and footwear industries is the strongest, indicating that HMs around LI industry sites represent the maximum level among the eight types.