Automatic gender classification from face images has been a popular topic among researchers for a decade. Feature extraction and classification methods are very important to create a successful automatic classification system. Due to the richness of face image datasets today, many successful machine learning and deep learning methods has been implemented. It is very critical to extract accurate features from the datasets to achieve promising classification scores when traditional machine learning methods are used. However, deep learning models have been designed to extract the features automatically from the raw data directly. This also automatize the feature extraction process besides classification. The hidden and unpredictable feature sets can be explored by the deep neural networks which can increase the classification performance comparing to traditional machine learning methods. Convolutional Neural Networks (CNN) as one of the effective classes of deep models have been adopted by many scientists for solving the gender classification problem. It can solve the problem of the fact that facial cues can change from origin to origin which makes the accurate feature extraction harder. There are several state-of-the art pretrained CNN structures which are very successful for image classification problems. The performance of CNNs is generally higher when the number of the input data is high. However, in this study, the success of the pretrained CNN models is investigated when the data is limited. Considering this fact, in this study, rather than using complete face images, only the one eye image regions with eyebrows are used for the gender classification. The performance results present that the best CNN models are NASNetLarge and Xception models.