Face classification is a challenging task that is crucial to numerous applications. There are many algorithms for classifying gender, but their ability to evaluate their effectiveness regarding scientific data is constrained. Deep learning is popular among researchers in face classification problems. The detection of many faces is complicated and becomes a necessity in real problems. The proposed research aims to examine the effect of twofold face detection approach on the accuracy of gender classification, as well as the effect of using small datasets on accuracy. In this study, we use a small dataset to classify facial images based on their gender. The following phases involve deep learning methods along with the OpenCV library version 3.4.2 which is recommended to serve as a twofold face detection approach. In the experiments conducted, Phase 1 is the designated training phase, and Phase 2 serves as a testing phase. Two different algorithms are used in the testing phase to detect one face in the image (Experiment 1), while the remaining algorithm detects multiple faces in the image (Experiment 2). The FEI dataset is used to evaluate the accuracy of the proposed research, which results in 84% accuracy for Experiment 2 and 74% for Experiment 1, respectively.