Recently, the security of individuals has become the prime concern for the human community. Various real-time security management systems are developed widely. Visual surveillance is considered as one of the promising technique to improve the security which helps to detect and recognize the objects. Numerous techniques have been developed for real-time video surveillance. Face detection, tracking and recognition is one of the important part of visual surveillance systems. The existing face detection schemes suffer from various challenging issues such as pose variations, illumination conditions and occlusion and many more. To overcome these issues, we have developed new schemes which includes Bayesian learning, Region Based Convolutional Neural Networks(RCNN) and GoogleNet based CNN model for face detection, tracking and recognition. In this work, we compare the performance of proposed schemes with existing schemes for different datasets to conclude the robustness of proposed approach.