The rapid and accurate detection of broccoli seedling planting quality is crucial for the implementation of robotic intelligent field management. However, existing algorithms often face issues of false detections and missed detections when identifying the categories of broccoli planting quality. For instance, the similarity between the features of broccoli root balls and soil, along with the potential for being obscured by leaves, leads to false detections of “exposed seedlings”. Additionally, features left by the end effector resemble the background, making the detection of the “missed hills” category challenging. Moreover, existing algorithms require substantial computational resources and memory. To address these challenges, we developed Seedling-YOLO, a deep-learning model dedicated to the visual detection of broccoli planting quality. Initially, we designed a new module, the Efficient Layer Aggregation Networks-Pconv (ELAN_P), utilizing partial convolution (Pconv). This module serves as the backbone feature extraction network, effectively reducing redundant calculations. Furthermore, the model incorporates the Content-aware ReAssembly of Features (CARAFE) and Coordinate Attention (CA), enhancing its focus on the long-range spatial information of challenging-to-detect samples. Experimental results demonstrate that our Seedling-YOLO model outperforms YOLOv4-tiny, YOLOv5s, YOLOv7-tiny, and YOLOv7 in terms of speed and precision, particularly in detecting ‘exposed seedlings’ and ‘missed hills’-key categories impacting yield, with Average Precision (AP) values of 94.2% and 92.2%, respectively. The model achieved a mean Average Precision of 0.5 (mAP@0.5) of 94.3% and a frame rate of 29.7 frames per second (FPS). In field tests conducted with double-row vegetable ridges at a plant spacing of 0.4 m and robot speed of 0.6 m/s, Seedling-YOLO exhibited optimal efficiency and precision. It achieved an actual detection precision of 93% and a detection efficiency of 180 plants/min, meeting the requirements for real-time and precise detection. This model can be deployed on seedling replenishment robots, providing a visual solution for robots, thereby enhancing vegetable yield.