This paper explores the principle of deep reinforcement learning algorithm application and constructs the digital teaching model of Civics through the recurrent neural network, Markov decision process, Actor-Critic algorithm and collaborative filtering recommendation algorithm. On this basis, the state representation model and decision-making model are invoked to improve the diversity optimization recommendation of the deep reinforcement learning algorithm. Aiming at the problems existing in the practical application of the OBE concept, the model constructed in this paper is utilized to propose a multi-platform integration of the Civics digital learning model. The objective of empirical and simulation experiments is to confirm the model’s implementation and recommendation effects, respectively. In the validation of hyperparameter d and N values, the DQN model achieves the optimal recommendation effect when the d values are 65 and 63 on the HetRec and MovieTweetings datasets. During the implementation of the digital teaching evaluation of the course Civics, more than 60% of the students rated the teacher’s feedback session higher. Overall, the 6 dimensions were rated higher, and the number of students who chose to be fully compliant and conforming was more than 200.