The blockage or failure of the drainage holes will endanger the stability of the slopes and traffic safety of a highway tunnel. This paper studies an algorithm for the automatic classification of drainage hole blockage degree based on convolutional neural network transfer learning to explore the intelligent detection method of drainage hole blockage. The model transfer method is adopted to input drainage hole image samples to retrain the pretrained network to classify new images. Experiments are performed on the collected samples of drainage hole images, and the accuracy of different network models is compared, ResNet-18 being the best. The ResNet-18 performance is compared using different transfer strategies and parameters. The results show that when the SGDM gradient optimisation algorithm is used and the learning rate is 0.0001, the identification effect of these samples is the best. The validation accuracy can reach 91.7%, test accuracy is 90.0%, and the effective classification of drainage hole blockage to different degrees is realised under the transfer learning strategy of ResNet-18 model 1–34 frozen layers. Furthermore, with an expansion of the samples in the future, the identification accuracy will be further improved. The automatic classification system of the blockage degree of drainage hole greatly reduces the cost of manual detection, plays a guiding role in the maintenance of drainage pipes, and effectively improves the safety of highway tunnels and slopes.