Across the globe, the transition towards sustainable energy systems necessitates seamless implementation of Renewable Energy Sources (RES) into traditional power grids. Such RESs include solar and wind power. The current research work intends to overcome the challenges associated with Optimal Power Flow (OPF) problem in power systems in which the traditional operation parameters ought to be optimized for effective and trustworthy integration of the RESs. The current study proposes an innovative nature-inspired approach by enhancing the Mayfly algorithm on the basis of mating behaviour of mayflies. The aim of this approach is to tackle the complexities introduced by dynamic and discontinuous nature of solar and wind power. The improved Mayfly algorithm aims at minimizing power losses, emission, optimize voltage profiles, and ensure reliable integration of solar and wind power. The current study outcomes provide knowledgeable insights towards power flow optimization in power systems with high penetration of renewable energy. The application results reveal that the improved mayfly algorithm achieved better efficacy compared to the classical mayfly algorithm and the rest of the optimization algorithms.