Intelligent Connected Vehicle (ICV), as a revolutionary technology for automobiles, is rapidly developing and changing the way people travel. However, the current smart cars lack the intelligent control of the powertrain, and even if the network connection is completed, the power, economy and emissions cannot be greatly improved. State-of-the-art deep reinforcement learning algorithms, whose agents continuously interact with the model, employ an end-to-end control strategy. The deep learning neural network is used to fit the mapping relationship between the state and the action, and the action of the agent is evaluated by the reinforcement learning reward function, and iteratively learns the control strategy that meets the goal. This paper adopts a new EGR control method based on deep reinforcement learning, and compares it with the traditional PID control method to verify whether the method is feasible and provide a reference for the intelligent control of the engine.