Determination of heavy metal (HM) contamination, ecological risks, and sources in river sediments are important to preventing and controlling environmental pollution. This study investigated the spatial distribution, potential ecological risks, and biological toxicity of five heavy metals in river sediments of Huanghua City in the water diversion area from the Yellow River, China. GIS, redundancy analysis (RDA), and the positive matrix factorization (PMF) model were used to accurately quantify the pollution sources and the spatial distribution of pollution sources. The results revealed that Cu had the highest degree of natural pollution, and the source mainly comes from traffic. Residential land (RL), population density (PD), GDP, and industrial construction (IC) make high contributions to traffic pollution; the highest level of potential ecological risk was Hg, and the source mainly comes from industrial wastewater discharges. IC makes a high contribution to industrial wastewater discharges pollution; the highest effect of bio-toxic risk was As, and the source mainly comes from farmland drainage water. Agricultural production potential (APP) and water area (WA) make high contributions to farmland drainage water pollution; Zn might be of natural origin, and woodlands (WLs) make high contribution to natural origin. This result provided a new idea for the system control of sediment heavy metal pollution in Huanghua City.