In this study, the focus was placed on the entire Hetao Basin that could fall into four hydrogeological units, i.e., Houtao Plain, Sanhuhe Plain, Hubao Plain, and South Bank Plain of Yellow River, which are under different geological and environmental conditions. To systematically investigate the hydrochemical characteristic and spatial distribution of high As groundwater (As >10 μg/L), 974 samples were collected from shallow groundwater. As indicated from the results, high As groundwater had an extensive distribution, and its spatial distribution in the four hydrogeological units exhibited significant variability. Three concentrated distribution areas were reported with high As groundwater, which were all in the discharge areas of groundwater, and the arsenic contents in groundwater was found to exceed 50 μg/L. The hydrochemical types of high As groundwater in the HT Plain and the SHH Plain consisted of HCO3 SO4·Cl by anions and Na by cations, while those in the other two plains included HCO3 by anions as well as Na·Mg·Ca by cations. According to the pH values, the groundwater was weakly alkaline in the areas with high As groundwater, and arsenic primarily existed as arsenite. Furthermore, high As groundwater in the Hetao Basin was characterized by high contents of Fe (mean value of 2.77 mg/L) and HCO3- (mean value of 460 mg/L) and a low relative concentration of SO42- (average value of 310 mg/L). This study did not identify any significant correlation in groundwater arsenic and other ions (e.g., Fe2+, Fe3+, HCO3-, SO42-, NO2- and NO3-) in the entire Hetao Basin and a wide range of hydrogeological units. The results remained unchanged after the four hydrogeological units were analyzed. The special sedimentary environment evolution of Hetao basin was found as the prerequisite for the formation of high arsenic groundwater. Furthermore, groundwater runoff conditions and hydrogeochemical processes in the basin were indicated as the factors controlling the formation of high arsenic groundwater.