In order to provide a new vision and a broad idea for the evaluation of physical education in colleges and universities, a physical education teaching effect evaluation system based on big data is proposed. This study uses the method of a questionnaire survey to understand the problems encountered in the current sports evaluation of a college in China and to determine the evaluation index and its weight coefficient of college sports. The foundation of big data applications is mainly used for the establishment of college sports evaluation. The results showed that 50 percent of physical education teachers and 44.2 percent of students thought physical education examination was unnecessary, while only 16.67 percent of physical education teachers and 24.2 percent of students thought it was necessary. However, 90 percent of physical education teachers and 90.4 percent of students think school physical education evaluation is very important. At the same time, 63.33% of P.E. teachers think that the evaluation of the college P.E. curriculum is not enough and there is no responsibility. At present, there are many problems in the evaluation of physical education courses in colleges and universities, such as the curriculum, model, and scale being simple and unclear, the process is not scientific, there being no good model, and the guarantee mechanism is not good. It is concluded that the current evaluation of physical education in colleges and universities in this province has many problems, such as single subject and model, simple and fuzzy index, unscientific method, lack of individuality in the standard, imperfect guarantee mechanism, and imperfect feedback mechanism.