Electrolytes are highly important for aqueous Al ion batteries (AAIBs). However, Al metal anode usually has poor reversibility of plating/tripping, resulting in low Coulombic efficiency (CE) and poor cycling stability by using traditional Al ion electrolyte. Herein, a novel type of aqueous Al ion electrolyte with polyethylene glycol (PEG) as the primary skeleton for solvated Al ions is proposed, named as PEG‐Al@H. Outstandingly, an Al electrolyte interface (AEI) is generated on the Al metal surface during galvanostatic charging process by polymerization of PEG. It is proved that AEI, acting as the protective layer, effectively mitigates the side reaction caused by the rapid kinetic in aqueous electrolyte and prevents Al metal anode from deeply corroding. Moreover, PEG disrupts the hydrogen bond of the solvent in the electrolyte, extending the working temperature of AAIBs. The design that a full‐cell consisting of PEG‐Al@H as the electrolyte, potassium manganese hexacyanoferrate as the cathode, Al metal as anode can run over 20 000 cycles at 500 mA g−1, with an average CE higher than 95%. More importantly, even at −5 °C, for the first time, an initial capacity of 16 mAh in a pouch‐cell is achieved and maintain roughly 87% of capacity after 5500 cycles.