Arable soils are crucial for national development and food security; therefore, contamination of agricultural soils from potentially toxic elements (PTEs) is a global concern. In this study, we collected 152 soil samples for evaluation. Considering the contamination factors and using the cumulative index and geostatistical methods, we investigated the contamination levels of PTEs in Baoshan City, China. Using principal component analysis, absolute principal component score-multivariate linear regression, positive matrix factorization, and UNMIX, we analyzed the sources and quantitatively estimated their contributions. The average Cd, As, Pb, Cu, and Zn concentrations were 0.28, 31.42, 47.59, 100.46, and 12.36 mg/kg, respectively. The Cd, Cu, and Zn concentrations exceeded the corresponding background values for Yunnan Province. The combined receptor models showed that natural and agricultural sources contributed primarily to Cd and Cu and As and Pb inputs, accounting for 35.23 and 7.67% pollution, respectively. Industrial and traffic sources contributed primarily to Pb and Zn inputs (47.12%). Anthropogenic activities and natural causes accounted for 64.76 and 35.23% of soil pollution, respectively. Industrial and traffic sources contributed 47.12% to pollution from anthropogenic activities. Accordingly, the control of industrial PTE pollution emissions should be strengthened, and awareness should be raised to protect arable land around roads.