Traditional recognition methods are simple to extract features and need to be manually extracted with high complexity and unstable accuracy. The expression recognition method of deep learning still has the problems of poor network representation ability and low recognition rate. In order to fully represent the complex texture and edge features of expression images, a deep learning method of expression recognition based on Gabor representation combined with PCNN was proposed. Firstly, different Gabor representations are obtained through a set of Gabor filter banks with different proportions and directions, and the corresponding convolutional neural network model is trained to generate G-CNNs. Then, the Pulse Coupled Neural Network (PCNN) was introduced to fuse the different outputs of G-CNNs. Experiments in CK+ and JAFFE databases show that the average recognition rates of this method obtained 94.87% and 96.91%. Compared with other methods, the proposed method achieves a better recognition effect.