Toxic metals accumulated in the human body are predominantly absorbed by the digestive tract in non-occupationally exposed populations. In the current study, we collected plowed soil samples, and investigated the varied food compositions in a mining and smelting area in southwestern China, to measure the concentrations of ten potentially toxic metals-As, Ba, Cd, Cr, Cu, Mn, Ni, Pb, Sr, and Zn. We collected information on the daily intakes of these metals to assess the health risks associated with their exposure among children and adults. The urine concentrations of the metals were also measured to obtain data on the amounts present in the body. The results showed that the hazard indexes (HIs) of As, Ba, Cd, Cr, Ni, Pb, and Sr were all larger than 1, for both adults and children; the Zn and Cu values were comparatively higher in children. The intake of staple foods of the region posed the greatest health risk, while drinking water only posed risks clearly associated with As exposure. The urine samples of local residents contained larger amounts of As, Cd, Cr, Cu, Mn, Ni, Pb, and Zn, confirming the hazards in the health risk assessment. Obvious differences in the urine metal concentrations between men and women were confirmed. In conclusion, higher concentrations of metals in the surface soil and rain water were the predominant cause of elevated exposure through home-grown crops and produce to accumulate in local residents' bodies.