In order to promote the teaching reform, deepen the professional construction, innovate the talent training mode and course teaching mode, improve teachers’ advanced vocational education teaching design ability and teaching level, and promote the connotation construction of higher vocational colleges, an evaluation method of higher mathematics teaching reform effect based on big data analysis technology is proposed. In the research, the teaching status of advanced mathematics courses and the problems existing in teaching are firstly analyzed and studied. Then, on the basis of comprehensive analysis of various influencing factors of course teaching effect evaluation, a set of two-level index system with a total of 25 indicators is established. The results verify the rationality of these indicator settings. Finally, combined with the real historical data of the teaching evaluation effect of a private college, the validity of the proposed model is verified. The experimental results show that the 530 sample data of the course are submitted to the RBF network for the training, and the network converges after 3385 iterations. Among all 100 test samples, the maximum test error is 0.0513, of which 92 samples are judged correct, 7 samples are judged as excellent, and 3 samples are judged as medium. Therefore, the evaluation accuracy rate of the RBF network is as high as 92%. It is concluded that this method provides a new way to evaluate the teaching effect of the course objectively and impartially.