Tangerine Peel has rich medicinal value, known as ' one kilogram of tangerine peel, one kilogram of gold '. However, the value of tangerine peels in different years is different, and there is no significant difference in the appearance of tangerine peels in different years. Identifying their authenticity has brought trouble to the industry. Generally speaking, the characteristics of tangerine peel can be identified through the texture, color and oil parcel points on the surface of tangerine peel. However, compared with the feature recognition of other Chinese medicinal materials, there is no significant difference in the shape of tangerine peel in different years, and the color is similar. Therefore, the feature extraction of tangerine peel is more complicated and the recognition is more difficult. The existing deep learning algorithms face great challenges in efficient and high accuracy recognition. In response to this challenge, this paper builds a new lightweight tangerine peel recognition algorithm TPRA (Tangerine Peel Recognition Algorithm) based on ResNet50. This algorithm uses a variety of methods to optimize the generalization ability of the model and improve the recognition accuracy. Firstly, TPRA adopts mixed data enhancement, including traditional data enhancement, deep convolution generation confrontation network DCGAN, and Mosaic data enhancement to enhance the richness of sample images in the dataset, reduced the data of each batch regularization (Batch Normal), and enhanced the performance of algorithm identification. Secondly, TPRA introduced the attention mechanism module CBAM (Convolutional Block Attention Module) combined with the cross stage partial network CSPNet (Cross Stage Partial Network) to propose an improved ResNet50 model, which adjusts the position of the maximum pooling layer and disassembles the large convolution kernel to effectively avoid overfitting. The experimental results showed that the accuracy of the algorithm can reach 98.8%, and the effect was better than that of Alexnet, VGG16 and Resnet50. TPRA provided a new method for the identification of peel years.