For a long time, temperature control and crack prevention of mass concrete is a difficult job in engineering. For temperature control and crack prevention, the most effective and common-used method is to embed cooling pipe in mass concrete. At present, there still exists some challenges in the precise simulation of pipe cooling in mass concrete, which is a complex heat-flow coupling problem. Numerical simulation is faced with the problem of over-simplification and inaccuracy. In this study, precise simulation of heat-flow coupling of pipe cooling in mass concrete is carried out based on finite element software COMSOL Multiphysics 5.4. Simulation results are comprehensively verified with results from theoretical solutions and equivalent algorithms, which prove the correctness and feasibility of precise simulation. Compared with an equivalent algorithm, precise simulation of pipe cooling in mass concrete can characterize the sharp temperature gradient around cooling pipe and the temperature rise of cooling water along pipeline more realistically. In addition, the cooling effects and local temperature gradient under different water flow (0.60 m3/h, 1.20 m3/h, and 1.80 m3/h) and water temperature (5 °C, 10 °C, and 15 °C) are comprehensively studied and related engineering suggestions are given.