A satellite altimetry mission can measure high-precision sea surface height (SSH) to recover a marine gravity field. The reference gravity field model plays an important role in this recovery. In this paper, reference gravity field models with different degrees are used to analyze their effects on the accuracy of recovering gravity anomalies using the inverse Vening Meinesz (IVM) method. We evaluate the specific performance of different reference gravity field models using CryoSat-2 and HY-2A under different marine bathymetry conditions. For the assessments using 1-mGal-accuracy shipborne gravity anomalies and the DTU17 model based on the inverse Stokes principle, the results show that CryoSat-2 and HY-2A using XGM2019e_2159 obtains the highest inversion accuracy when marine bathymetry is less than 2000 m. Compared with the EGM2008 model, the accuracy of CryoSat-2 and HY-2A is improved by 0.6747 mGal and 0.6165 mGal, respectively. A weighted fusion method that incorporates multiple reference models is proposed to improve the accuracy of recovering gravity anomalies using altimetry satellites in shallow water. The experiments show that the weighted fusion method using different reference models can improve the accuracy of recovering gravity anomalies in shallow water.