Gender classification provides additional information about the individual's identity, which is crucial in surveillance, smart interface, and smart advertising, it is a very important aspect of face analysis that has piqued the interest of researchers in areas such as demographic information collection, surveillance, human-computer interaction, marketing intelligence, security, etc. Usually, facial images are used to extract features for classification. This study aims to determine gender based on the entire face or the eyes (for masked or occlusion faces). The proposed method consists of six main stages: (1) Remove the background using the Deep Labelling Version 3 plus (DeepLabV3+) method; (2) Skin detection by applying the combination of the two colors spaces models (Hue, Saturation, Value (HSV)) and (Luminance, Chrominance blue and Chrominance red (YCbCr)); (3) Face detection using the Haar Cascades classifier method; (4) Face alignment and cropping; (5) Classify the gender. Classification of the gender when the image contains the entire face is based on a deep wavelet. In the case of occlusion faces, we proposed to use a Convolution Neural Network (CNN) for classifying the gender from the eye (s). The dataset used for training this model is the celeb faces attributes dataset (CelebA) and some of the different datasets were also used to compare this model with the other previous works. In addition, we showed that our models perform well with images with some challenges, for example, some of the faces are not fully visible, not completely frontal, wearing glasses with different styles, have low quality and noise, have various lighting, children's and infants' faces, with and without makeup, open and closed mouths and closed eyes. Other than that, the accuracy achieved was 98% when classifying from face images and 98% from the eye. The proposed method was compared with previous works and was very promising. The contributions of this proposal were the use of the deep wavelet for gender classification and the proposal of a new method for skin detection, which enhances the performance of the Haar cascade method used for face detection. Also, we proposed a method for face alignment and, finally, the classification of genders with many challenges from the entire face or just from the eye (s).