Video anomaly detection refers to the rapid identification and localization of anomalous events during video surveillance. It has become a challenging task due to the dependence of anomalous event definitions on specific scenes and the scarcity, diversity and non-exhaustiveness of anomalous events. This manuscript proposes a robust frame prediction-based method for video anomaly detection, it can solve the problems of underutilized spatio-temporal sequence information and difficulty in capturing subtle motions. Firstly, a spatial-temporal feature fusion network is constructed using FusionNet and LSTM. Secondly, feature optimization is performed on this spatio-temporal feature fusion network using optical flow feature enhancement. To further improve the robustness of the model, patch GAN is added to FusionNet-LSTM for adversarial training, and the objective function in training is optimized with apparent constraints and image similarity constraints to approach the real frame. Finally, to reduce the sequence error, a PNSR-based mean normalized anomaly score calculation is used in the manuscript. Experiments show that the average accuracy of the proposed method for anomalous event detection on UCSD Ped2 and CUHK Avenue datasets is 96.9% and 86.1% respectively, which is 1.5% and 1.0% better than the benchmark model U-Net，and the model has low computational complexity where FLOP is 6.196G and the number of parameters is 6.829M. It can meet the application requirements of real-time video anomaly detection and has great value for security video surveillance.