Three carbon fibers (CF) of different characteristics and properties were used to determine the most suitable characteristics for a carbon fiber to reinforce a thermoplastic polyurethane elastomer (TPU) matrix. The TPU was synthesized with a moderate amount of hard segments to bring polarity to the matrix without losing flexibility. On the other hand, the CF were chosen with respect to their different mechanical properties, diameter, and length to study the influence of those parameters on the TPU matrix. Additionally, the fibers were used in different amounts as well as after different oxidation treatments (i.e., different polarity). The mechanical properties and surface characteristics of the composites were analyzed using several experimental techniques (SEM, X-ray photoelectron spectroscopy, dynamic mechanical thermal analysis, and tensile tests). The results obtained showed that the surface polarity, diameter, and length of the CFs are related to the reinforcement of a medial polar TPU matrix, whereas the mechanical properties of the CF do not influence those of the composite.