For the design of ground-source heat pumps (GSHPs), two design parameters, namely the ground thermal conductivity and borehole thermal resistance are estimated by interpreting thermal response test (TRT) data using a physical model. In most cases, the parameters are fitted to the measured data assuming that the chosen model can fully reproduce the actual physical response. However, two significant sources of error make the estimation uncertain: random error from experiments and structural bias error that describes the discrepancy between the model and actual physical phenomena. Generally, these two error sources are not evaluated separately. As a result, the suitability of selected models to correctly infer parameters from TRTs are not well understood. In this study, the Bayesian calibration framework proposed by Kennedy and O'Hagan is employed to estimate the GSHP design parameters and quantify the random and structural errors in the inference. The calibration framework enables us to examine structural errors in the commonly used infinite line source model arising due to the conditions in which the TRT takes place. Two in situ TRT datasets were used: TRT1, influenced by contextual disturbances from the outdoor environment, and TRT2, influenced by a strong groundwater flow caused by heavy rainfall. We show that the Bayesian calibration framework is able to quantify the structural errors in the TRT interpretation and therefore can yield more accurate estimates of design parameters with full quantification of uncertainties.