In this paper, Local Binary Patterns (LBP) and their derivatives, like Local Ternary Patterns (LTP), Local Gradient Patterns (LGP), Non-Redundant Local BinaryPatterns (NRLBP) and multi-scale images processed by LBPs, are evaluated in order to find the optimal features for the automatic face recognition system. The comparison of LBP and its variations is performed based on the recognition accuracy. The genetic algorithm optimizes a criterion function, which combines four parameters, such as LBP feature type, feature image processing type, and feature dimension and distance measure. The evaluation was performed on four different face databases. The proposed methodology can be applied in various kinds of recognition, such as facial expression recognition. The main strength of this paper is the design methodology for the selection of the most discriminative features, in accordance with the desired feature vector length and face recognition accuracy.