It is highly uncertain as to the potential risk of toxic metal(loid)s in abandoned mine soil. In this study, random forest was used to predict the risk of cadmium pollution in the soils of an abandoned lead/zinc mine. The results showed that the random forest model is stable and precise for the pollution risk prediction of toxic metal(loid)s. The mean of Cd, Cu, Tl, Zn, and Pb was 6.02, 1.30, 1.18, 2.03, and 2.08 times higher than the soil background values of China, respectively, and their coefficients of variation were above 30%. As a case study, cadmium in the mine soil had “slope” hazard characteristics while the ore sorting area was the major source area of cadmium. The theoretical values of the random forest model are similar to the practical values for the ore sorting area, metallogenic belt, riparian zone, smelting area, hazardous waste landfill, and mining area. The potential risk of soil Cd in the ore sorting area, metallogenic belt, and riparian zone are extremely high. The tendency of pollution risk migrates significantly both from the ore sorting area to the smelting area and the mining area, and to the hazardous waste landfill. The correlation of soil pollution risk is significant between the mining area, the smelting area, and the riparian zone. The results suggested that the random forest model can effectively evaluate and predict the potential risk of the spatial heterogeneity of toxic metal(loid)s in abandoned mine soils.