Precise prediction of future vehicle information can improve the control efficiency of hybrid electric vehicles. Nowadays, most prediction models use previous information of vehicles to predict future driving velocity, which cannot reflect the impact of the driver and the environment. In this paper, a real-time energy management strategy (EMS) based on driver-action-impact MPC is proposed for series hybrid electric vehicles. The proposed EMS consists of two modules: the velocity prediction module and the real-time MPC module. In the velocity prediction module, a long short-term memory (LSTM) neural network model, which is trained by the traffic data derived from a VR-based driving simulator, is adopted to predict the future driving information by using driver action information and current vehicle’s velocity. The obtained future driving velocity is treated as the inputs of the real-time MPC module, which outputs the control variables to act on the underlying controllers of power components by solving a standard quadratic programming (QP) problem. Compared with the rule-based strategy, a 5.6% average reduction of fuel consumption is obtained. The effectiveness of real-time computation of the EMS is validated and verified through a hardware-in-the-loop test platform.