In the Xining area of Huangshui River, China, the concentrations of globally alarming toxic heavy metals such as chromium (Cr), arsenic (As), lead (Pb), cadmium (Cd), and zinc (Zn) in the surface sediments were measured to determine the ecological risks to the riverine ecosystem. Overall, the concentrations of heavy metals in the surface sediments were relatively high. The results indicated that the concentrations of Pb/Zn/Cr were lower than 120/250/200 mg·kg−1 which was the risk screening value of each heavy metal for soil contamination of agricultural land (GB15618-2018). The concentrations of As/Cd were higher than 120/3.0 mg·kg−1 which was the risk intervention value of As/Cd for soil contamination of agricultural land (GB15618-2018). Their values were arranged in the following decreasing order: As (151.23–818.55 mg·kg−1) > Cr (44.18–201.70 mg·kg−1) > Pb (29.10–121.95 mg·kg−1) > Zn (1.45–86.18 mg·kg−1) > Cd (4.36–6.21 mg·kg−1). The concentrations of As, Cr Pb, and Cd greatly exceeded the background contents of elements in Qinghai soils. While Zn was lower than background. The contamination factor (CF), the geoaccumulation index (Igeo), and the potential ecological risk index (RI) of As, Cr, Cd, and Pb followed the descending order of Cd > As > Pb > Cr > Zn. The order of the enrichment factor (EF) was as follows: As > Cd > Pb > Cr > Zn. The contamination level of heavy metals implied that the condition is frightening and probably severely affecting the aquatic ecology. Cd and As were the main contributing elements to the ecological risk of the sediments in the Xining area of the Huangshui River, which should be mainly studied and prevented.