The purpose is to effectively solve the problems of high time cost, low detection accuracy, and difficult standard training samples in video processing. Based on previous investigations, football game videos are taken as research objects, and their shots are segmented to extract the keyframes. The football game videos are divided into different semantic shots using the semantic annotation method. The key events and data in the football videos are analyzed and processed using a combination of artificial rules and a genetic algorithm. Finally, the performance of the proposed model is evaluated and analyzed by using concrete example videos as data sets. Results demonstrate that adding simple artificial rules based on the classic semantic annotation algorithms can save a lot of time and costs while ensuring accuracy. The target events can be extracted and located initially using a unique lens. The model constructed by the genetic algorithm can provide higher accuracy when the training samples are insufficient. The recall and precision of events using the text detection method can reach 96.62% and 98.81%, respectively. Therefore, the proposed model has high video recognition accuracy, which can provide certain research ideas and practical experience for extracting and processing affective information in subsequent videos.