Fiber-reinforced polymer (FRP) rods are advanced composite materials with high strength, light weight, non-corrosive properties, and superior durability properties. Under severe environmental conditions, for concrete structures, the use of glass-fiber-reinforced polymer (GFRP) rods is a cost-effective alternative to traditional steel reinforcement. This study compared the flexural behavior of an OPC concrete slab with a high-volume fly ash (HVFA) concrete slab reinforced with GFRP rods/steel rods. In the fly ash concrete slabs, 60% of the cement used for casting the slab elements was replaced with class F fly ash, which is emerging as an eco-friendly and inexpensive replacement for ordinary Portland cement (OPC). The data presented include the crack pattern, load–deflection behavior, load–strain behavior, moment–curvature behavior, and ductility of the slab specimens. Additionally, good agreement was obtained between the experimental and nonlinear finite element analysis results using ANSYS 2022-R1. The study also compared the experimental moment capacity with the most commonly used design standard ACI 440.1R-15. This investigation reveals that there is a huge potential for the utilization of GFRP rods as reinforcement in fly ash concrete slabs.