Solid-state polymer electrolytes are considered to be the most promising electrolytes for next-generation high-energy rechargeable lithium batteries due to the advantages of high safety, good mechanical flexibility, and easy film-formation ability. Among all the polymers, polyethylene oxide (PEO) is demonstrated to be a feasible polymer host, based on its high dielectric constant and strong lithium salt dissolving ability. However, the practical application of PEO in the all-solid-state lithium batteries is limited mainly by its low ionic conductivity at room temperature. For decades, researchers dedicate to increase the ion conductivity at room temperature and mechanical properties according to the technology strategy of composite polymer electrolytes. In particular, the electrode/electrolyte interface structure is designed and optimized according to the requirement of different battery systems. Accordingly, in this review, the basic characteristics, ion transport mechanism, composite mechanism of inert/active fillers with polymers, and electrode/ electrolyte interface structures are evaluated for the PEO-based composite polymer electrolytes. Finally, the outlook is presented for future development of the solid-state polymer electrolytes and high-energy rechargeable lithium batteries.