This study describes the development of a new class of high-performance polyurethane elastomer nanocomposites containing reduced graphene oxide (RGO) or graphene nanoplatelets (GNP). Two types of polyurethane elastomers with different contents of hard segments (HS) were used as a polymer matrix. The developed nanocomposites were characterized by thermal analysis (DSC, TG), dynamic mechanical testing (DMA), hardness testing, mechanical properties, rheology, FTIR spectroscopy, XRD, and microscopy investigation (TEM, SEM). Morphological investigation confirmed better compatibility of RGO with the polyurethane (PU) matrix compared to GNP. Both applied nanofillers influenced melting and crystallization of the PU matrix. The nonlinear viscoelastic behavior of the nanocomposites (Payne effect) was studied, and the results were compared with theoretical predictions.