Eco-driving control generates significant energysaving potential in car-following scenarios. However, the influence of preceding vehicle may impose unnecessary velocity waves and deteriorate fuel economy. In this research, a learning-based method is exploited to achieve satisfied fuel economy for connected plug-in hybrid electric vehicles (PHEVs) with the advantage of vehicle-to-vehicle communication system. A data-driven energy consumption model is leveraged to generate reinforcement signals for approximate dynamic programming (ADP) with the consideration of nonlinear efficiency characteristics of hybrid powertrain system. An advanced ADP scheme is designed for connected PHEVs driving in car-following scenarios. Additionally, the cooperative information is incorporated to further improve the fuel economy of the vehicle under the premise of driving safety. The proposed method is mode-free and showcases acceptable computational efficiency as well as adaptability. The simulation results demonstrate that the fuel economy during car-following processes is remarkably improved through cooperative driving information, thereby partially paving the theoretical basis for energy-saving transportation.