Renewable energies are clean alternatives to the highly polluting fossil fuels that are still used in the power generation sector. The goal of this research was to look into replacing a Heavy Fuel Oil (HFO) thermal power plant in Limbe, southwest Cameroon, with a hybrid photovoltaic (PV) and wind power plant combined with a storage system. Lithium batteries and hydrogen associated with fuel cells make up this storage system. The total cost (TC) of the project over its lifetime was minimized in order to achieve the optimal sizing of the hybrid power plant components. To ensure the reliability of the new hybrid power plant, a criterion measuring the loss of power supply probability (LPSP) was implemented as a constraint. Moth Flame Optimization (MFO), Improved Grey Wolf Optimizer (I-GWO), Multi-Verse Optimizer (MVO), and African Vulture Optimization Algorithm (AVOA) were used to solve this single-objective optimization problem. The optimization techniques entailed the development of mathematical models of the components, with hourly weather data for the selected site and the output of the replaced thermal power plant serving as input data. All four algorithms produced acceptable and reasonably comparable results. However, in terms of proportion, the total cost obtained with the MFO algorithm was 0.32%, 0.40%, and 0.63% lower than the total costs obtained with the I-GWO, MVO, and AVOA algorithms, respectively. Finally, the effect of the type of storage coupled to the PV and wind systems on the overall project cost was assessed. The MFO meta-heuristic was used to compare the results for the PV–Wind–Hydrogen–Lithium Battery, PV–Wind–Hydrogen, and PV–Wind–Lithium Battery scenarios. The scenario of the PV–Wind–Hydrogen–Lithium Battery had the lowest total cost. This scenario’s total cost was 2.40% and 18% lower than the PV–Wind–Hydrogen and PV–Wind–Lithium Battery scenarios.