The use of cement-based composites (CBC) with high thermal conductivity for geothermal well cementing is extremely important for the efficient development and use of geothermal energy. Accurate prediction of thermal conductivity can save a lot of experimental costs and time. At present, there is no specific calculation model for the thermal conductivity of CBC. In this study, the microstructure, thermal conductivity model and influencing factors of CBC were investigated by experimental tests, theoretical analysis and numerical simulation. The results showed that the cement-based material could be simplified into a two-layer structure of hydrated and unhydrated layers. Mathematical and numerical models based on the coupled Series model and the Maxwell–Eucken model were established to calculate the thermal conductivity for CBC. The mathematical and numerical models were found to be more accurate by comparison with the conventional models and experimental test results. The cubic packing was more favorable than the spherical packing to improve the thermal conductivity of CBC. The plate material had significant anisotropy. The thermal conductivity of CBC showed a rapid decrease followed by a slow decrease, a decrease followed by a slow increase and finally a rapid decrease, a rapid increase followed by an up and down fluctuation and finally a plateau, respectively, with the increase of filler particle diameter, spacing and curing temperature. Based on these results, the effective methods and future research directions were proposed to maximize the thermal conductivity of geothermal well cementing materials in actual engineering applications. The research findings can provide some technical references for the efficient development of geothermal energy and research on CBC with high thermal conductivity.