The influence of surface modification on the properties of bagasse fibers and asphalt binders/mixtures was investigated. Bagasse fibers were modified by single, binary, and ternary methods with hydrochloric acid, sodium hydroxide, and sodium chlorite, respectively. The physical and chemical properties of bagasse fibers were analyzed by scanning electron microscopy, Fourier transform infrared spectroscopy, and an adsorption test, respectively. The rheological properties of asphalt binders with bagasse fibers or lignin fibers were analyzed by the dynamic shear rheometer test and bending beam rheometer test. In addition, the performance of asphalt mixtures with bagasse fibers or lignin fibers were evaluated by a wheel rutting test, bending test at a low temperature, and water stability test, respectively. In conclusion, the hydrophilic functional groups on the fiber surface were partially eliminated by modification, facilitating the degradation of different fiber components. Furthermore, the degree of fibrillation was improved, and more interfaces with asphalt components were formed, thus enhancing the high-temperature deformation resistance of asphalt binders, but slightly impairing its low-temperature performance. Among all modification methods, the ternary composite modification exerted important influences on fiber structure, oil absorption, and rheological properties of asphalt binders, significantly enhancing the performance of asphalt mixtures. Combined with surface modification methods, bagasse fibers would be promising reinforced pavement materials.