For ground source heat utilization systems, pile heat exchangers are sometimes used. In order for these systems to achieve high performance, control of the system dynamics is important, and the underground temperature must be known. Typically, underground temperature is measured using a thermometer in a borehole. However, in the case of pile heat exchangers, a different method is required, making the system expensive to set up. To overcome this problem, the installation of underground thermometers in the heat exchanger piles themselves is proposed in the present study. The proposed thermometer system consists of thermocouples packed in grout such as silica sand within the piles. However, there is a possibility of measurement errors due to vertical thermal conduction in the steel pipes, and it is important to estimate the measurement accuracy before the development of this system. In the present study, the measurement accuracy is estimated using numerical simulations and then confirmed experimentally. The underground temperature profiles inside and outside the pile are compared. The results indicate that the proposed system offers sufficient accuracy for application to pile heat exchangers.