Fiber materials as an asphalt mixture additive and stabilizer can effectively improve the performance index of asphalt pavement. In this study, lignin and carbon fiber were used as modifiers to study their effects on the road performance of asphalt mastic. Based on the frequency sweep, linear amplitude sweep (LAS) and multi-stress creep recovery (MSCR) experiments were conducted to test the high-temperature rutting and medium-temperature fatigue resistance of asphalt mastic with different fiber incorporation and low-temperature performance tests based on bending beam rheometer (BBR). The results indicate that adding fibers increased the stiffness of the asphalt mastic, and the modification effect of lignin fibers was better than that of carbon fibers. Meanwhile, the characteristic flow index of the asphalt mastic gradually increased with the increase in temperature, indicating that it gradually became a near-Newtonian fluid at higher temperatures. The addition of fibers also improved the high temperature rutting resistance of the asphalt mastic but did not have an advantageous effect on fatigue and low temperature cracking resistance. Additionally, the fitting results of the four-parameter Burgers model show that the use of fiber modification decreases the proportion of elasticity and viscous creep compliance but increases the delayed elasticity part.