In this paper, a heat recovery ground source heat pump (HR-GSHP) system, in which the primary pipes of the GSHP for air conditioning and the GSHP for hot water are connected to ground heat exchangers (GHEs) and each GSHP is operated simultaneously or within a short period of time, was installed in a dormitory building on a trial basis. Then, the optimal operation method to minimize the energy consumption of the system was investigated. The operating period of the GSHP for HW was changed and simulations were conducted to determine the operating period with the lowest energy consumption, which was 8 months from April to November. Furthermore, the HR-GSHP system was operated for 8 years from 2012 to 2019, and actual measurements were carried out to verify the system performance and the energy saving effect in optimal operation. In actual operation, it was confirmed that the minimum temperature was about 10 °C or higher even when the GSHP for HW was operated year-round. Therefore, the GSHP for HW was operated year-round after the third year of operation. It was confirmed that the operation of the GSHP for HW in summer, especially in August and September when the cooling load is large, can improve the system’s efficiency by the effect of recovering cooling exhaust heat. In the eighth year of operation, when the GSHP for HW was operated most during the summer season, the system was able to reduce power consumption for air conditioning and hot water supply by approximately 17%.