This paper presents an experimental study about the influence of interfacial properties on the performance of fiber-reinforced asphalt. In this study, four types of fiber including one fiber-reinforced plastic (FRP), two lignin fibers, and one basalt fiber are used, and also four types of asphalt: Asphalt No. 90, asphalt No. 70, one styrene-butadiene-styrene (SBS) modified asphalt, and asphalt rubber are used. The surface energy parameters of various asphalts and fibers and the shear strength of various fiber-reinforced asphalts are measured. On the basis of these measurements, the influences of surface properties of asphalt and fiber on the performance of fiber-reinforced asphalt are analyzed. The results show that the shear strength of asphalt binder can be significantly increased by adding fibers, and the reinforcement effect is closely related to the types of asphalt and fiber. It was discovered, for the same asphalt, that the basalt fiber has the best reinforcement effect, followed by the two lignin fibers, and the FRP. For the same fiber, asphalt rubber was the most reinforced, followed by the SBS modified asphalt, asphalt No. 70 and asphalt No. 90. It was also discovered, for the same asphalt, the higher the surface energy of the fiber, the better the fiber reinforcement effect. The analysis indicates a good correlation between the work of adhesion between asphalt and fiber and the effect of fiber reinforcement. The results can be used as a basis for the selection of the proper fiber-asphalt combination to improve fiber reinforcement effects.