Real-time, rapid, accurate, and non-destructive batch testing of fruit growth state is crucial for improving economic benefits. However, for plums, environmental variability, multi-scale, occlusion, overlapping of leaves or fruits pose significant challenges to accurate and complete labeling using mainstream algorithms like YOLOv5. In this study, we established the first artificial dataset of plums and used deep learning to improve target detection. Our improved YOLOv5 algorithm achieved more accurate and rapid batch identification of immature plums, resulting in improved quality and economic benefits. The YOLOv5-plum algorithm showed 91.65% recognition accuracy for immature plums after our algorithmic improvements. Currently, the YOLOv5-plum algorithm has demonstrated significant advantages in detecting unripe plums and can potentially be applied to other unripe fruits in the future.