Despite wide applications of reduced graphene-oxide (GO)-reinforced polymer-based composites, the necessity of the reduction procedure toward GO is still controversial. In this article, thermoplastic polyurethane (TPU) composites incorporated with GO and thermally reduced graphene oxide (TGO) were fabricated. GO and TGO exhibited different effects on crystallization behaviors, and mechanical and thermal properties of the TPU matrix. With 2.0 wt % filler loading, TPU composite reinforced by GO (TPU-GO-2 wt %) exhibited better thermal stability than that reinforced by TGO (TPU-TGO-2 wt %). The interfacial interaction between the nanofillers and the TPU matrix as well as their influence on the mobility of TPU chains were investigated, which proved that GO is superior to TGO in improving interface adhesion and maintaining crystallization of the TPU matrix. Compared with TPU-TGO-2 wt %, improved mechanical properties of TPU-GO-2 wt % were also evidenced owing to more oxygen-containing groups. This work demonstrates that the reduction of GO is not always necessary in fabricating polymer composites.