In issues pertaining to higher education, deep learning (DL), and its connection to educational data, it is crucial to forecast students’ success. The ability to predict a student’s success aids in choosing courses and developing future study schedules. Apart from forecasting children’s performance, it also assists educators and administrators in keeping an eye on pupils, offering them support, and incorporating training initiatives to maximize outcomes. Student prediction has the advantage of lowering official warning flags and removing ineffective pupils from the classroom. By helping students select courses and study schedules that are suited for their skill levels, prediction supports the students directly. In the proposed approach, a methodology based on the integration of convolutional neural network (CNN) and long short-term memory (LSTM) is proposed to optimize students’ performance prediction systems. This study utilized a student performance dataset from the UCI ML Repository. It includes information on student achievements in secondary education from two Portuguese schools. The suggested method overcomes the following three problems in model development: an imbalanced dataset, a lack of feedback mechanism to enhance the quality of learning, and an inadequate mechanism to extract the learning patterns/relevant features to predict student performance. The suggested system’s effectiveness has been demonstrated by the accuracy (98.45) and loss (0.1989) obtained to achieve the best prediction.