A total of 43 water and sediment samples, and 34 Corbicula fluminea samples were collected in Xijiang River in southern China to determine the spatial distribution and sources of 12 metals/metalloids (V, Co, Cr, Ni, Cu, Mn, Zn, Cd, Pb, As, Sb, and Tl) and to assess the pollution levels and ecological risks of the pollutants. The results showed that the levels of the metals/metalloids (except for Tl) in the river water from almost all of the sampling sites met the Chinese national surface water quality standards. However, the concentrations of the metals/metalloids in the sediments exceeded the background values by a factor of 1.03–56.56 except for V, Co, and Mn, and the contents of Zn, Cd, and Pb in the Corbicula fluminea soft tissue exceeded the limits of the Chinese Category I food Quality Standards. The spatial distribution analysis showed that the concentrations of the contaminants in the lower reaches of Xijiang River were higher than in the upper reaches. The bioaccumulation factor (BAF), biota-sediment accumulation factor (BSF), geo-accumulation index (Igeo), and the potential ecological risk index (RI) were obtained to assess the pollution levels and ecological risks. The results indicated that Cu, Cd, and Zn were the most prone to bio-accumulation in the Corbicula fluminea soft tissue, and the lower reaches showed a much higher pollution level and risk than the upper reaches. The metals/metalloids in the sediments posed serious threat on the aquatic ecosystem, of which Cd, As, and Sb are the most risky contaminants. The results of principal component analysis (PCA) indicated Cr, Ni, Cu, Mn, Cd, Pb, and As in the sediments came from relevant industrial activities, and V and Co originated from natural sources, and Sb from mining activities, Zn and Tl came from industrial activities and mining activities.