Interface interactions play a crucial role in determining the thermomechanical properties of carbon nanotubes (CNTs)/polymer nanocomposites. They are, however, poorly treated in the current multi-scale coarse-grained (CG) models. To develop suitable CG models of CNTs/polymer nanocomposites, we demonstrate the importance of two aspects for the first time, that is, preserving the interfacial cohesive energy and reproducing the interface load transfer behavior of all-atomistic (AA) systems. Our simulation results indicate that, for CNTs/polymer nanocomposites, the interface cohesive energy and the interface load transfer of CG models are generally inconsistent with their AA counterparts, revealing significant deviations in their predicted mechanical properties. Fortunately, such inconsistency can be “corrected” by phenomenologically adjusting the cohesive interaction strength parameter of the interface LJ potentials in conjunction with choosing a reasonable degree of coarse-graining of incorporated CNTs. We believe that the problem studied here is general for the development of the CG models of nanocomposites, and the proposed strategy used in present work may be applied to polymer nanocomposites reinforced by other nanofillers.