There are various public and private places such as banks, roads, offices, and homes equipped with cameras for surveillance. The surveillance videos are consisting of a precious source of information related to critical application scopes. The main problem is to aid powerful and accessible software that changes the content present in the video for the forgery creation of a video. The forgery involves region duplication that has a common video tampering. The existing techniques are utilized to detect video tampering from the forged videos that showed complexity in the background. Thus, it is important to overcome the problem of forgery detection in the research. The Spatio-temporal averaging model is carried out for the collection of a video sequence for obtaining the background information. This can detect the moving objects effectively for forgery detection. Next, the ResNet 18 is used for extraction of the feature vectors, and the discriminative feature vectors were reduced and improved the training time and accuracy. The Single Auto Encoder (SAE) is not able to reduce the input features' dimensionality. Thus, the SAE has used 3 encoders stacked on the top for detecting the forgery. It is based on the sequence of videos. In comparison to the existing models, the proposed approach outperformed them with accuracy rates of 98.6%, sensitivity rates of 98.60%, specificity rates of 98.47%, MCC rates of 97.29%, and precision rates of 99.93%.