To effectively use (Li) lithium metal anodes, it is becoming increasingly necessary to create membranes with high lithium conductivity, electrochemical and thermal stabilities, as well as adequate mechanical properties. Composite gel polymer electrolytes (CGPE) have emerged as a promising strategy, offering improved ionic conductivity and structural performance compared to polymer electrolytes. In this study, a simple and scalable approach was developed to fabricate a crosslinked polyethylene oxide (PEO)-based membrane, comprising two different glass fiber reinforcements, in terms of morphology and thickness. The incorporation of a solvated ionic liquid into the developed membrane enhances the ionic conductivity and reduces flammability in the resulting CGPE. Galvanostatic cycling experiments demonstrate favorable performance of the composite membrane in symmetric Li cells. Furthermore, the CGPE demonstrated electrochemical stability, enabling the cell to cycle continuously for more than 700 h at a temperature of 40 °C without short circuits. When applied in a half-cell configuration with lithium iron phosphate (LFP) cathodes, the composite membrane enabled cycling at different current densities, achieving a discharge capacity of 144 mAh·g−1. Overall, the findings obtained in this work highlight the potential of crosslinked PEO-based composite membranes for high-performance Li metal anodes, with enhanced near room temperature conductivity, electrochemical stability, and cycling capability.