Zanthoxyli Pericarpium (ZP) are the dried ripe peel of Zanthoxylum schinifolium Sieb. et Zucc (ZC) or Zanthoxylum bungeanum Maxim (ZB). It has wide range of uses both medicine and food, and favorable market value. The diverse specifications of components of ZP is exceptional, and the common aims of adulteration for economic profit is conducted. In this work, a novel method for the identification different species of ZP is proposed using convolutional neural networks (CNNs). The data used for the experiment is 5 classes obtained from camera and mobile phones. Firstly, the data considering 2 categories are trained to detect the labels by YOLO. Then, the multiple deep learning including VGG, ResNet, Inception v4, and DenseNet are introduced to identify the different species of ZP (HZB, DZB, OZB, ZA and JZC). In order to assess the performance of CNNs, compared with two traditional identification models including Support Vector Machines (SVM) and Back Propagation (BP). The experimental results demonstrate that the CNN model have a better performance to identify different species of ZP and the highest identification accuracy is 99.35%. The present study is proved to be a useful strategy for the discrimination of different traditional Chinese medicines (TCMs).