In the external drainage water of Xinjiang Karamay Petrochemical Company, the calcium and magnesium ion contents are as high as 630 mg/L and 1170 mg/L, respectively. These ions are subsequently scaled at large quantities in water treatment equipment, which greatly reduces the efficiency of the process. This study used a coupled chemical precipitation (CP) and electrocoagulation (EC) method to deal with external drainage water. The results show that the removal rates of calcium and magnesium were 94.71% and 96.33%, respectively, when the pH was adjusted with sodium silicate and EC was introduced after saturating the water samples with CO2. In this study, a quadratic polynomial equation was developed for predicting the removal of calcium and magnesium ions using this coupled technique under three factors of current density (CD) (15, 20 and 25 mA/cm2), reaction time (RT) (30, 40 and 50 min) and pH (10.0, 10.5 and 11.0). It was found that these three variables have a significant effect on the removal of both the abovementioned ions. The response surface method based on a Box–Behnken design showed that the average removal rates of calcium and magnesium ions could be 96.57% and 98.66% at a CD of 22 mA/cm2, RT of 46 min and pH of 11. This study confirmed the presence of calcium carbonate in the solid product through XRD and SEM analysis. The results indicate that this study is promising, and the developed technique can also be used to remove the high concentrations of calcium and magnesium ions from different wastewaters.