The energy management strategy has a great influence on the fuel economy of hybrid electric vehicles, and the equivalent consumption minimization strategy (ECMS) has proved to be a useful tool for the real-time optimal control of Hybrid Electric Vehicles (HEVs). However, the adaptation of the equivalent factor poses a major challenge in order to obtain optimal fuel consumption as well as robustness to varying driving cycles. In this paper, an adaptive-ECMS based on driving pattern recognition (DPR) is established for hybrid electric vehicles with continuously variable transmission. The learning vector quantization (LVQ) neural network model was adopted for the on-line DPR algorithm. The influence of the battery state of charge (SOC) on the optimal equivalent factor was studied under different driving patterns. On this basis, a method of adaptation of the equivalent factor was proposed by considering the type of driving pattern and the battery SOC. Besides that, in order to enhance drivability, penalty terms were introduced to constrain frequent engine on/off events and large variations of the continuously variable transmission (CVT) speed ratio. Simulation results showed that the proposed method efficiently improved the equivalent fuel consumption with charge-sustaining operations and also took into account driving comfort.