In this work, to reduce the probability of brittle failure in the support structure of deeply buried high-stress soft rock roadways, hybrid-fiber reinforced rubber concrete (HFRRC) was investigated using the orthogonal test, and the effects of various factors on the performance were studied. The mechanical properties, pore structure, and microstructure of rubber concrete reinforced by basalt fiber (BF) and polyvinyl alcohol fiber (PF) were studied from macroscale, mesoscale, and microscale perspectives. The results revealed that the content of the rubber particles has a significant impact on strength. Further, the addition of the hybrid fibers to the concrete was found to have a positive effect on the splitting tensile strength and the flexural strength. However, no significant effect was observed on the compressive strength. Furthermore, it was found that the content of BF and PF have a significant impact on the energy dissipation capacity and ductility, and the influence of the PF content is greater than that of the BF content. The concrete with 10% rubber particles of 1–3 mm, a volume fraction 0.3% basalt fiber, and a volume fraction 0.2% polyvinyl alcohol fiber was obtained as the optimal mix proportions. Moreover, it was found that the random distribution of the rubber particles and the hybrid fibers optimized the pore structure, inhibited the expansion of the cracks, and reduced the brittleness of the concrete. The findings of this study can provide a useful reference for the application of an environmentally friendly material with recycled rubber aggregate and hybrid fiber.