Pose and illumination variation in Face Recognition (FR) is a problem of fundamental importance in computer vision. We propose to tackle this problem by using Chirp Z-Transform (CZT) and Goertzel algorithm as preprocessing, Block-based feature extraction and Exponential Binary Particle Swarm Optimization (EBPSO) for feature selection. Every stage of the FR system is examined and an attempt is made to improve each stage. The unique combination of CZT and Goertzel algorithm is used for illumination normalization. The proposed feature extractor uses a unique technique of Block based Additive Fusion of the image. EBPSO is a feature selection algorithm used to select the optimal feature subset. The proposed approach has been tested on four benchmark face databases, viz., Color FERET, HP, Extended Yale B and CMU PIE datasets, and demonstrates better performance compared to existing methods in the presence of pose and illumination variations.