In the course of asphalt pavement usage, exposure to fuel infiltration accelerates particle detachment, leading to the occurrence of problems such as looseness and peeling. The aim of this study was to comprehensively evaluate the impact of a fuel-resistant modifier (FRM; 1%, 3%, 5%, 7%) on the performance of asphalt and asphalt mixture. Conventional physical tests and high–low temperature rheological tests were conducted on the fuel-resistant modified asphalt (FRMA). The results indicate that, with increased FRM content, the penetration and ductility of FRMA decreased, while the softening point increased. The high-temperature performance improved, but the low-temperature ductility declined. High–low temperature rheological test results demonstrate that the addition of FRM significantly enhanced the asphalt’s shear deformation resistance. A moderate amount of FRM modification improved the asphalt’s low-temperature crack resistance, but excessive FRM resulted in reduced flexibility. In addition, fuel-resistant modified asphalt mixture (FRMAM) specimens were prepared and evaluated for performance. In comparison to the base asphalt, FRM modification enhanced the resistance to oil erosion and peeling as well as the Marshall stability of asphalt mixtures. It reduced the scattering loss caused by fuel dissolution and improved both low-temperature indirect tensile (IDT) strength and high-temperature shear strength. A low content of FRM effectively reduced the sensitivity of asphalt mixtures to fuel infiltration, thus enhancing their road performance.