Automotive industries are constantly seeking to produce more powerful and efficient engines. One of the factors affecting engine efficiency is the engine's temperature. Several research studies have been done to control and reduce engine temperature. The generated heat due to combustion of fuel in the engine should be cooled down to avoid engine heat up. In the present study, the cooling circuit of a real‐scale internal combustion engine is simulated using GT‐suite software along with a proposed model. Also, in this study, a model for simulating boiling and the effect of employing nanofluids as a coolant is presented. Initially, the performance of the proposed model is examined in a channel. The results show that the proposed model increases accuracy by about 20% compared with the default GT‐suit model, which ignores boiling. The simulation results show that the presence of Al2O3 and TiO2 nanofluids up to 2 vol% reduces the engine wall temperature by 7% and 6.7%, respectively.