Incorporating fibers into asphalt mixtures as additives and stabilizers can significantly enhance the performance of asphalt pavements. This study aimed to analyze the impact of using basalt and bamboo fibers as modifiers on the properties of asphalt mastics. The effects of different types of fibers on rutting resistance, fatigue resistance, elastic recovery, and low-temperature cracking performance were tested using frequency scanning, linear amplitude scanning (LAS), multiple stress creep and recovery (MSCR), elastic recovery, and bending beam rheometer (BBR) experiments. The study results suggest that adding fibers into asphalt mastics can effectively improve their stiffness, and the higher the fiber content, the better the stiffness enhancement. Moreover, the characteristic flow index of asphalt mastics grows gradually with the rise in temperature, indicating that these materials exhibit near-Newtonian fluid behavior at elevated temperatures. Furthermore, incorporating fibers significantly enhances the high-temperature rutting resistance of asphalt mastics. However, the addition of fibers did not demonstrate any appreciable benefits in terms of fatigue resistance. The elasticity of asphalt mastics cannot be significantly changed by fiber content without compromising their elastic recovery. Surprisingly, the study’s findings showed that adding basalt fibers to asphalt mastics did not improve their resistance to low-temperature cracks. On the other hand, it was discovered that the ability of asphalt mastics to resist cracking at low temperatures could be made up for by the use of bamboo fibers as a modifier together with a raised temperature. Overall, it was discovered that bamboo fibers performed better than basalt fibers at improving the performance of modified asphalt mastics.