Predicting the damage to a concrete panel under impact loading is difficult due to the complexity of the impact mechanism of concrete. Based on the experimental results obtained by various researchers, the energies involved in the impact mechanism are classified into seven categories: Kinetic energy, deformed energy of a projectile, elastic penetration resistance energy of the panel, overall deformed energy of the panel, spalling-resistant energy, tunneling-resistant energy, and scabbing-resistant energy. Using these impact mechanisms and the energy conservation law, a new energy-based penetration depth formula is proposed to predict the penetration depth. This is validated using 402 impact test results, which include those with high-strength concrete, ultra-high-performance concrete (UHPC), or steel fiber-reinforced concrete, those under very high-velocity impact, and those with a very low ratio of target panel thickness to projectile diameter. It is found that the new impact formula predicts the penetration depth quite well.