Based on the groundwater chemistry data from Karamay City, Xinjiang Province, this study examines seven major components: K+, Ca2+, Na+, Mg2+, SO42−, Cl−, and HCO3−. The sampling was conducted during two periods: flood and dry periods. On the basis of analyzing the regional geologic background and hydrogeologic conditions, the study area was divided into calculation units and the test data validity was screened. Then, the outliers were eliminated by Grubbs’ method and Piper’s trilinear plot method, and the effectiveness of the elimination was evaluated by the box-and-line diagram. After that, the distribution types of the groundwater chemistry data in each calculation unit were judged to obtain the background values of the seven test indexes for the groundwater in different calculation units. The results show that Grubbs’ method and Piper’s trilinear plot method are effective in removing outliers. Secondly, the background values of Na+ and Cl− in the groundwater of Karamay City are mostly higher than those of other anions and cations during the two sampling periods. This phenomenon may be due to the concentration effect of evaporation. Additionally, the groundwater background values of Ca2+, Na+, and Cl− showed more significant differences between the two sampling periods compared to other ions.