Star anise, a widely popular spice, benefits from classification that enhances its economic value. In response to the low identification efficiency and accuracy of star anise varieties in the market, as well as the scarcity of related research, this study proposes an efficient identification method based on non-similarity augmentation and a lightweight cascaded neural network. Specifically, this approach utilizes a Siamese enhanced data network and a front-end SRGAN network to address sample imbalance and the challenge of identifying blurred images. The YOLOv8 model is further lightweight to reduce memory usage and increase detection speed, followed by optimization of the weight parameters through an extended training strategy. Additionally, a diversified fusion dataset of star anise, incorporating open data, was constructed to further validate the feasibility and effectiveness of this method. Testing showed that the SA-SRYOLOv8 detection model achieved an average detection precision (mAP) of 96.37%, with a detection speed of 146 FPS. Ablation experiment results showed that compared to the original YOLOv8 and the improved YOLOv8, the cascade model’s mAP increased by 0.09 to 0.81 percentage points. Additionally, when compared to mainstream detection models such as SSD, Fast R-CNN, YOLOv3, YOLOv5, YOLOX, and YOLOv7, the cascade model’s mAP increased by 1.81 to 19.7 percentage points. Furthermore, the model was significantly lighter, at only about 7.4% of the weight of YOLOv3, and operated at twice the speed of YOLOv7. Visualization results demonstrated that the cascade model accurately detected multiple star anise varieties across different scenarios, achieving high-precision detection targets. The model proposed in this study can provide new theoretical frameworks and ideas for constructing real-time star anise detection systems, offering new technological applications for smart agriculture.