Autonomous driving technology is one of the important methods that avoid the hidden dangers of traffic safety. Although the existing autonomous driving technology can meet some needs of real traffic scenarios, the ability of multi-agents to effectively generate autonomous driving strategy in complex traffic environments remains to be improved. Aiming at this problem, the automatic drive model based on game theory and reinforcement learning is proposed by combining these two technologies and applying them in multi-agent cooperative driving, which enables multi-agents to carry out strategic reasoning with negotiation in traffic scenarios by extending the game description language, and puts forward the constrained multi-agents deep deterministic policies gradient algorithm. Finally, the related experiments are conducted through the autonomous driving simulation platform, and the experimental results show that the proposed model can effectively generate driving strategies for multi-agents in complex traffic environments, which verifies the validity and feasibility of the proposed model, and provides the general research basis for multi-agents autonomous driving.