The agricultural soil alongside highways has experienced multiple potential ecological risks from human activities. In this study, 100 soil samples near the highways were collected in Lishui District, Nanjing City. Using the single-factor pollution index, the Nemerow comprehensive pollution index, and the potential ecological risk index, the study investigated the heavy metal contents and distribution in roadside agricultural soil. PCA and a multiple regression model were applied to quantitatively analyze the spatial relationships between sampling soil heavy metal accumulation and the surrounding man-made landscape. The mean contents of Cu and Pb exceeded the background, while Cd, Cr, and Zn were lower than that. The potential ecological risk index exhibited a very low ecological hazard and only Cr in soils rarely showed moderate risk. Furthermore, quantitative analysis for the sources of contamination revealed that agricultural practices were the dominant contributors to the heavy metals, including Cd, Cu, and Zn, while road and heavy industrial practices contributed to Cr and Pb. The study provides sources of heavy metal pollution from human activities in roadside agricultural land and serves as a reference for ecological restoration.