Heavy metal pollution in farmland soil represents a considerable risk to ecosystems and human health, constituting a global concern. Focusing on a key area for the cultivation of special agricultural products in Cangxi County, we collected 228 surface soil samples. We analyzed the concentration, spatial distribution, and pollution levels of six heavy metals (Cr, Cu, Pb, Ni, Zn, and Hg) in the soil. Moreover, we investigated the sources and contribution rates of these heavy metals using Principal Component Analysis/Absolute Principal Component Scores (PCA/APCS) and Positive Matrix Factorization (PMF) models. Our findings indicate that none of the six metals exceeded the pollution thresholds for farmland soils. However, the mean concentrations of Cr and Ni surpassed the background levels of Sichuan Province. A moderate spatial correlation existed between Pb and Ni, attributable to both natural and anthropogenic factors, whereas Zn, Cu, Hg, and Cr displayed a strong spatial correlation, mainly due to natural factors. The spatial patterns of Cr, Cu, Zn, Pb, and Ni were similar, with higher concentrations in the northern and eastern regions and lower concentrations centrally. Hg’s spatial distribution differed, exhibiting a broader range of lower values. The single pollution index evaluation showed that Cr and Ni were low pollution, and the other elements were no pollution. The average value of comprehensive pollution index is 0.994, and the degree of pollution is close to light pollution. Predominantly, higher pollution levels in the northern and eastern regions, lower around reservoirs. The PCA/APCS model identified two main pollution sources: agricultural traffic mixed source (65.2%) and natural parent source (17.2%). The PMF model delineated three sources: agricultural activities (32.59%), transportation (30.64%), and natural parent sources (36.77%). Comparatively, the PMF model proved more accurate and reliable, yielding findings more aligned with the study area’s actual conditions.