Security has always been a significant concern since the dawn of human civilization. That is why we build houses to keep ourselves and our belongings safe. And we do not hesitate to spend a lot on front-door locks and install CCTV cameras to monitor security threats. This paper presents an innovative automatic Front Door Security (FDS) algorithm that uses Human Activity Recognition (HAR) to detect four different security threats at the front door from a real-time video feed with 73.18% accuracy. The activities are recognized using an innovative combination of GoogleNet-BiLSTM hybrid network. This network receives the video feed from the CCTV camera and classifies the activities. The proposed algorithm uses this classification to alert any attempts to break the door by kicking, punching, or hitting. Furthermore, the proposed FDS algorithm is effective in detecting gun violence at the front door, which further strengthens security. This Human Activity Recognition (HAR)-based novel FDS algorithm demonstrates the potential of ensuring better safety with 71.49% precision, 68.2% recall, and an F1-score of 0.65.