In this paper, the ultra-high drop hammer impact test system is adopted for the vertical impact experiment on six SFSRCBs with steel fiber volume fraction of 2.0% and build the relevant numerical model. The research variables mainly include impact energy, impact mass and cumulative impact mode. The results show that, under a single equal energy impact, with the increase of the impact speed, the crack distribution of the specimen gradually tends to the mid-span local position. Compared with the impact mass, the impact speed has a greater effect on the failure mode, deformation resistance and deformation recovery ability of SFSRCBs. When the total cumulative impact energy is the same as the single impact energy, then the overall damage of the specimen caused by high-mass low-velocity cumulative impact is less significant than that caused by low-mass high-velocity cumulative impact. Combined with the finite element analysis, the calculation formula of the maximum deflection of SFSRCB under impact load is obtained. Finally, the calculation method of the impact damage evaluation factor of SFSRCBs is proposed.