The objectives of this study were to understand the characteristics of heavy metal pollution caused by mining activities on the two sides of the Shun’an river and the response of soil microorganisms to the habitats by different contamination levels and vegetation. This paper selected soil samples from the banks of the Shun’an River near the Shizishan mining area, which is at the left of the river, in Tongling, Anhui Province, China. Using Illumina MiSeq 2500 technology, we analyzed the relationship between environmental factors and microbial communities. As the distance from the mining area increased, the heavy metal comprehensive pollution and potential risk value decreased. Additionally, the pollution severity and risk value of the left bank, where the mining area lies, were generally higher than those of the right bank. Because the symmetric sampling points on both banks of the river had similar planting types, their environmental factors and microbial community structure were similar and clustered. However, under different vegetation, the paddy soils tended to have a higher nutrient content and community richness and diversity than the vegetable fields or the abandoned land. It was found that soil microbial communities in this area were mostly affected by pH and Nemerow pollution index (PN). The pH significantly affected the abundance and structure of most microorganisms. In addition, Proteobacteria, Acidobacteria, and Bacteroidetes had significant tolerance to Zn, Pb, and Cd. By exploring the potential use of these tolerant microorganisms, we seek to provide strains and the theoretical basis for the bioremediation of areas contaminated by heavy metal.