Summary Ground source heat pumps have high efficiency and high capital cost primarily due to borehole drillings. This research investigates the inclusion of high‐conductivity phase change material (PCM) in the borehole heat exchanger of a ground source heat pump to reduce the borehole length required and improve its coefficient of performance (COP). In the laboratory model, the borehole heat exchanger was represented by a cylindrical electrical heater having a total power of 9.216 W, operating for 1 hour while resting for 3 hours. Surrounding the heater in the annular region, either soil, PCM, or high‐conductivity PCM was used as grouting material. The annular region was surrounded by a large amount of soil enclosed in a large bin as a representation of ground soil. The high‐conductivity graphite was impregnated with the commercial PCM “PureTemp29.” Results from the experiments revealed that the PCM is able to decrease the temperature fluctuations in the annular and soil regions, while graphite increases the thermal conductivity of the annular region and hence increases the rate of heat dissipation from the heater to the soil surrounding it. The maximum COP values of a ground source heat pump calculated assuming ideal reversed Carnot cycle for cooling mode showed an increase of approximately 81% with PCM and by 112% with graphite‐enhanced PCM.