Safe and effective mining of phosphate rock plays an important role in the sustainable development of phosphorus resources. The mechanical properties and failure process of phosphate rock under different mining rates remain unclear, further restricting the safe and efficient mining of phosphate rock. In this paper, infrared radiation and uniaxial compression tests of phosphate rock under different loading rates, e.g., 0.0005, 0.001, 0.005, and 0.01 mm/s, were conducted to investigate the failure process of phosphate rock. Energy evolution and infrared radiation characteristics of the phosphate rock damage process were analyzed using nondestructive, real-time, and noncontact infrared thermal imaging technology. The results show that the higher the loading rate of phosphate rock, the more obvious the high-temperature zone and high-temperature point of phosphate rock in the loading process. At failure, the friction and slippage between internal cracks are intense, showing that the infrared radiation temperature difference increases with the increase of loading rate. As loading rate increases, the energy release time of phosphate rock before damage is reduced, resulting in more energy stored in the rock as evinced by its infrared radiation characteristics, finally resulting in greater damage. The increase of loading rate reduces the dissipation energy of phosphate rock before failure so that more energy remains in the rock mass through the weak surface of the grain boundary. The results of this work will be helpful in enhancing theoretical support for prevention and control of dynamic disasters in phosphate mines.