The center symmetric pattern (CSP) was widely used in the local binary pattern based facial feature, whereas never used to develop the illumination invariant measure in the literature. This paper proposes a novel diagonal symmetric pattern (DSP) to develop the illumination invariant measure for severe illumination variation face recognition. Firstly, the subtraction of two diagonal symmetric pixels is defined as the DSP unit in the face local region, which may be positive or negative. The DSP model is obtained by combining the positive and negative DSP units in the even × even block region. Then, the DSP model can be used to generate several DSP images based on the 2 × 2 block or the 4 × 4 block by controlling the proportions of positive and negative DSP units, which results in the DSP2 image or the DSP4 image. The single DSP2 or DSP4 image with the arctangent function can develop the DSP2-face or the DSP4-face. Multi DSP2 or DSP4 images employ the extended sparse representation classification (ESRC) as the classifier that can form the DSP2 images based classification (DSP2C) or the DSP4 images based classification (DSP4C). Further, the DSP model is integrated with the pre-trained deep learning (PDL) model to construct the DSP-PDL model. Finally, the experimental results on the Extended Yale B, CMU PIE, AR, and VGGFace2 face databases indicate that the proposed methods are efficient to tackle severe illumination variations. INDEX TERMS Severe illumination variations, diagonal symmetric pattern, center symmetric pattern, single sample face recognition.