The thermophysical properties of backfill material (BM) in a heat exchange borehole significantly influence the heat exchange effect of ground source heat pumps (GSHPs). Several treatments such as compaction and adding bentonite, cement, and fine sands are often used to improve the thermophysical properties. In this study, a 3A molecular sieve (3A-MS), a type of porous material, was added to the BM to enhance its water maintaining capacity. Three types of backfill materials with different additive contents, named as BM-0, BM-1, and BM-2, were examined. The variation of the BM properties such as the soil–water characteristic curve (SWCC), thermal conductivity, specific heat capacity, and thermal diffusivity with the groundwater content were investigated through a series of experiments and simulations. A scanning electron microscope (SEM), an energy dispersive spectrometer (EDS), and the BET method for specific surface area pore size analysis were used to characterize the material. The results indicated that the specific heat capacity improved with the water content whereas the thermal conductivity and thermal diffusivity decreased with the water content. The variation of the buried pipe outlet temperature with the change of the thermal physical parameters of the BM were researched by a numerical simulation and theoretical calculations; the results showed that BM-2 could raise the heat transfer rate per meter by 45.9% in summer and 118.4% in winter compared with the backfill materials without groundwater (NW). The research results provide theoretical support for the improvement of BM for ground source heat pump projects where abundant groundwater is available.