Recently, recognition of gender from facial images has gained a lot of importance. There exist a handful of research work that focus on feature extraction to obtain gender specific information from facial images. However, analyzing different facial regions and their fusion help in deciding the gender of a person from facial images. In this paper, we propose a new approach to identify gender from frontal facial images that is robust to background, illumination, intensity, and facial expression. In our framework, first the frontal face image is divided into a number of distinct regions based on facial landmark points that are obtained by the Chehra model proposed by Asthana et al. The model provides 49 facial landmark points covering different regions of the face, e.g. forehead, left eye, right eye, lips. Next, a face image is segmented into facial regions using landmark points and features are extracted from each region. The Compass LBP feature, a variant of LBP feature, has been used in our framework to obtain discriminative gender specific information. Following this, a Support Vector Machine based classifier has been used to compute the probability scores from each facial region. Finally, the classification scores obtained from individual regions are combined with a genetic algorithm based learning to improve 2 Avirup Bhattacharyya et al.the overall classification accuracy. The experiments have been performed on popular face image datasets such as Adience, cFERET (color FERET), LFW and two sketch datasets, namely CUFS and CUFSF. Through experiments, we have observed that, the proposed method outperforms existing approaches.The face of a human being reveals useful information about his/her identity, ethnicity, age, gender, expression, and emotion. Gender identification from facial images plays an important role in various computer vision based applications. It is possible for human beings to correctly determine gender by looking at the facial appearance. The advancements of computer vision technologies have inspired the researchers to design systems capable of performing similar functions [15,26,30]. Human computer interaction system, surveillance system, content based indexing and searching, biometric, and targeted advertising are some of the areas where it has been widely used. In a typical face recognition system, researchers face a number of challenges due to variations in pose, illumination, occlusion, expression, and lower resolution that essentially hamper a system's performance. Hence, the challenge is to design a system that is fairly invariant to changes in illumination, pose, resolution, and facial expression. These challenges have been effectively dealt with by the authors in [20,27,36]. However, gender recognition in such varying conditions has not been fully addressed yet. This paper deals with gender recognition of frontal face images with variations in illumination, pose, facial expression and image resolution.Feature dimension and computational complexity are important aspects that a typical gen...