Currently, multi-layer deep convolutional networks are mostly used for field weed recognition to extract and identify target features. However, in practical application scenarios, they still face challenges such as insufficient recognition accuracy, a large number of model parameters, and slow detection speed. In response to the above problems, using chickweed as the identification object, a weed identification model based on improved YOLOv5s was proposed. Firstly, the Squeeze-and-Excitation Module (SE) and Convolutional Block Attention Module (CBAM) were added to the model’s feature extraction network to improve the model’s recognition accuracy; secondly, the Ghost convolution lightweight feature fusion network was introduced to effectively identify the volume, parameter amount, and calculation amount of the model, and make the model lightweight; finally, we replaced the loss function in the original target bounding box with the Efficient Intersection over Union (EloU) loss function to further improve the detection performance of the improved YOLOv5s model. After testing, the accuracy of the improved YOLOv5s model was 96.80%, the recall rate was 94.00%, the average precision was 93.20%, and the frame rate was 14.01 fps, which were improved by 6.6%, 4.4%, 1.0%, and 6.1%, respectively, compared to the original YOLOv5s model. The model volume was 9.6 MB, the calculation amount was 13.6 GB, and the parameter amount was 5.9 MB, which decreased by 29.4%, 14.5%, and 13.2% compared with the original YOLOv5s model, respectively. This model can effectively distinguish chickweed between crops. This research can provide theoretical and technical support for efficient identification of weeds in complex field environments.