Recently, carbon materials with unique properties, such as high tensile strength and electrical conductivity, have been extensively investigated for the multi-functionalization of concrete. Previous studies mainly focused on improving the performance of normal-strength concrete using carbon nanomaterials, such as carbon nanotubes and graphene nanoplates. Therefore, this study analyzed the effect of carbon materials on ultra-high-performance concrete (UHPC) mixed with steel fibers, which has an outstanding mechanical performance. In addition, length effects were investigated for carbon fibers with nanometer, micrometer, and millimeter sizes. The influences of carbon materials on 120 MPa UHPC were investigated, including expanded graphite, a well-known superior conductivity material. Electrical conductivity, compressive strength, tensile strength, and electrical conductivity were analyzed experimentally. As a result, compressive strength tends to decrease as the concentrations of carbon materials increase, and chopped fiber has the best performance at 10.5 MPa in terms of tensile strength. Since the electrical conductivity of chopped fiber was observed to be significantly higher than that of other materials at 6.6 times, millimeter-sized fiber would be most suitable as a carbon material for concrete. This study could guide future research on the multi-functionalization of UHPC with carbon-based materials, including mechanical and electrical conductivity performances.