Using low-temperature (shallow) groundwater as a heat source or heat sink is a common practice to supply space heating or cooling, especially in the United States, Canada, China, and several European countries. The groundwater heat pump (GWHP) system has been extensively studied in recent decades using numerical approaches, which have some limitations in understanding the soil’s thermal behavior. Therefore, a laboratory-scale experimental study involving cooling tests was carried out to investigate the impact of GWHP on system performance and sustainability with varying groundwater flow velocities and injection and abstraction rates. The results demonstrated that groundwater flow velocity, as well as injection and abstraction rates, significantly impact thermal plume development. Higher injection and abstraction rates create a larger thermal plume, thereby decreasing abstraction temperature. However, groundwater flow prevents heat development around the well by dispersing the heat in the groundwater flow direction. Furthermore, the results indicate that the energy gain only increased by 81% and 107%, with a respective increase of 100% and 200% in injection and abstraction rates.