During the evolution of the twenty-first century, the increase in people’s material life level has also brought an increase in mental stress. For college students, not only need to bear heavy academic pressure, but also face the market competition after entering the society. Therefore, the psychological fitness instruction for undergraduate students has become an inseparable part of college education. However, in the current stage of mental health education, the quality of teaching is often difficult to guarantee. For the purpose of promoting the in-depth implementation of mental health education in colleges and universities and improving teaching effectiveness, one needs to analyze the quality of teaching in great details and understand the factors affecting the quality of teaching. In this paper, the mathematical programming algorithm is used to analyze the quality of college students’ Psychological Wellness Instruction. It deeply studies the development status of psychological wellness training and the basic composition and algorithm principle of mathematical programming algorithm. It integrates it into the practice of teaching quality analysis on the basis of linear regression algorithm theory. The experimental results show that the highest confidence interval of teachers’ teaching organization, cognitive level, and students’ participation level in the mental health course has reached more than 80.00%. This shows the importance of these three factors to the improvement of teaching quality. In order to enhance the quality of college students’ psychological wellness training, it is necessary to innovate the form of classroom organization and the degree of interaction between students and the classroom.