Real-time security requirements continue to increase due to the occurrence of various suspicious activities in open and closed environments. Day-today security threats may seriously affect everyone lives. Many techniques have been introduced in this regard, but still some issues remain unaddressed. The work presented in this paper provides video surveillance with improved accuracy and less computational complexity. The most significant part of the system consists of face localization, detection and recognition. The system obtains underlined facial data through a video dataset or from the real-time environment. Subsequently, face/foreground and background keyframes are extracted from at hand captured video data. Finally, extracted facial image data is compared with the facial images in the database. In case no match is found with the existing data, a security alarm or signal is generated, alerting security personnel to take action. The proposed system is more accurate, has better performance, and is low cost compared with existing systems. INDEX TERMS Face recognition, HOG features, feedforward backpropagation neural network, surveillance video, principal component analysis.