Farmland heavy metal pollution—caused by both human activity and natural processes—is a major global issue. In the current study, principal component analysis (PCA), cluster analysis (CA), rare earth elements and yttrium (REY) analysis, and isotope fingerprinting were combined to identify sources of heavy metal pollution in soil from different farmland types in the upper-middle area of the Yangtze River. The concentrations of Zn and Cu were found to be higher in the vegetable base and tea plantation soil compared with their concentrations in the orangery soil. On the other hand, greater accumulation of Cd and Pb was observed in the orangery soil versus the vegetable base and tea plantation soils. Influenced by the type of bedrock, REY was significantly enriched in the orangery soil and depleted in the vegetable base soil, as compared with the tea plantation soil. The Pb isotopic compositions of the tea plantation (1.173–1.193 for 206Pb/207Pb and 2.070–2.110 for 208Pb/206Pb) and vegetable base (1.181–1.217 for 206Pb/207Pb and 2.052–2.116 for 208Pb/206Pb) soils were comparable to those of coal combustion soil. The compositions of 206Pb/207Pb (1.149–1.170) and 208Pb/206Pb (2.121–2.143) in the orangery soil fell between those observed in soils obtained from coal combustion and ore smelting sites. Using the IsoSource model, the atmospheric Pb contributions of the vegetable base, tea plantation, and orangery soils were calculated to be 66.6%, 90.1%, and 82.0%, respectively, and the bedrock contributions of Pb were calculated to be 33.3%, 9.90%, and 18.1%, respectively. Based on the PCA, CA, and REY results, as well as the Pb isotope model, it appears that heavy metals in the orangery soil may be derived from atmospheric deposition and bedrock weathering, while heavy metals in the vegetable base and tea plantation soils may be derived from mining and the use of fertilizer.