With the growing emphasis on sustainable development in the construction industry, fiber-reinforced recycled aggregate concrete (BFRC) has attracted considerable attention due to its superior mechanical properties and environmental benefits. However, accurately predicting the compressive strength of BFRC remains a challenge because of the complex interaction between recycled aggregates and fiber reinforcement. This study introduces an innovative predictive framework that combines the XGBoost machine learning algorithm with advanced optimization algorithms, including the Seagull Optimization Algorithm (SOA), Tunicate Swarm Algorithm (TSA), and Mayfly Algorithm (MA). The unique integration of these algorithms not only improves predictive accuracy but also optimizes model performance by enhancing parameter tuning capabilities. Experimental results demonstrated that the TSA-XGBoost model achieved an exceptional R2 of 0.9847 and a minimum mean square error (MSE) of 0.255958, outperforming other models in predicting BFRC’s compressive strength. This novel predictive approach offers an efficient and accurate tool for assessing BFRC’s mechanical performance in practical applications, thus supporting its broader adoption in sustainable construction.