To develop the utilization of pasture fibers and waste tire rubber powder, the effect of different blending levels of modified pasture fibers (2 kg/m3, 3 kg/m3, and 4 kg/m3) on the loss rate of compressive toughness and softening characteristics of rubber powder concrete was studied. In addition, RapidAir457 system imaging analysis and microscopic electron microscope photo analysis were carried out to analyze the distribution pattern of modified pasture fibers and rubber powder in concrete. The results show that the peak load of concrete is higher than other pasture fiber dosing test groups when 3 kg/m3 is incorporated in 20 mesh and 60 mesh rubber powder. With the increase of pasture fiber content, the compressive toughness loss of MC60 group and MC80 group increased first and then decreased. The softening curve analysis showed that MC-3 was more capable of absorbing damage energy in the strain range of 0.0005–0.0009 than the MC-2 group. According to the RapidAir457 system imaging analysis, the pasture fibers form a web-like organization inside the concrete when the pasture fiber is mixed at 3 kg/m3.