To solve the problems of weak generalization of potato early and late blight recognition models in real complex scenarios, susceptibility to interference from crop varieties, colour characteristics, leaf spot shapes, disease cycles and environmental factors, and strong dependence on storage and computational resources, an improved YOLO v5 model (DA-ActNN-YOLOV5) is proposed to study potato diseases of different cycles in multiple regional scenarios. Thirteen data augmentation techniques were used to expand the data to improve model generalization and prevent overfitting; potato leaves were extracted by YOLO v5 image segmentation and labelled with LabelMe for building data samples; the component modules of the YOLO v5 network were replaced using model compression technology (ActNN) for potato disease detection when the device is low on memory. Based on this, the features extracted from all network layers are visualized, and the extraction of features from each network layer can be distinguished, from which an understanding of the feature learning behavior of the deep model can be obtained. The results show that in the scenario of multiple complex factors interacting, the identification accuracy of early and late potato blight in this study reached 99.81%. The introduced data augmentation technique improved the average accuracy by 9.22%. Compared with the uncompressed YOLO v5 model, the integrated ActNN runs more efficiently, the accuracy loss due to compressed parameters is less than 0.65%, and the time consumption does not exceed 30 min, which saves a lot of computational cost and time. In summary, this research method can accurately identify potato early and late blight in various scenarios.