In viscoelastic turbulent wall-bounded flows, the suppression of near-wall vortical structures due to viscoelastic stress significantly reduces both the frictional drag and heat transfer. To investigate the effect of the Prandtl number (Pr) on the heat transfer reduction rate (HTR), we conducted a series of direct numerical simulations of passive scalar transport using the finitely extensible nonlinear elastic-Peterlin (FENE-P) model for a viscoelastic turbulent channel flow. Various values of Pr from 0.1 to 5.0 were tested at a frictional Reynolds number of 125. The results revealed that the HTR was almost constant for Pr≥2.0 at a given drag-reduced flow and was higher than the drag reduction rate, aligning with previous experimental observations. However, in the case of lower-Pr fluids (Pr≤0.7), the HTR decreased as Pr decreased. The variation in the Nusselt number (Nu) for Pr was examined by decomposing Nu into three components: laminar flow contribution, turbulent heat flux contribution, and contribution owing to the deviation in the mean velocity profile from the laminar profile. For lower-Pr fluids (Pr≤0.7), the contribution of the wall-normal turbulent heat flux was insufficient to achieve the same HTR as that observed for Pr = 5.0. Despite the reduced wall-normal turbulent heat flux in the viscoelastic flows, the instantaneous flow fields showed a substantial similarity in the turbulent structures of the Reynolds shear stress compared to those of the wall-normal turbulent heat flux, which was maintained at various Pr values. This was also statistically confirmed through the weighted joint probability density function.