In this study, the correlation between the microstructure of straw fibers and the macroscopic properties of asphalt binders is investigated. Penetration, softening point, bending beam rheometer, and toughness tests are performed to investigate the relationship between the pavement properties of fiber asphalt binders and each of the fiber types and contents. Subsequently, a four-component test, SEM, Fourier transform infrared microscopy, and fluorescence microscope tests are performed to analyze the microstructure of the straw fiber and its effect on asphalt properties. The results show that the high-temperature performance and shear resistance of the asphalt binder were enhanced with an increase in the modified straw fiber content. The low-temperature performance and toughness of the asphalt binder were reduced because of excessive fiber, and the recommended fiber content is 1.5%–2.0%. The adsorption capacity of the modified straw fiber for the light components of asphalt increased by 5.4% on average, and the low-temperature creep capacity of asphalt decreased by 9.6%. The surface roughness of the straw fiber increased via modification, and the shear resistance, high-temperature deformation resistance, and stress relaxation ability of the asphalt increased by 23.1%, 6.5%, and 5.7%, respectively. The comprehensive properties of the asphalt binder with modified straw fiber are similar to those of lignin fiber.