The real-time target detection of crop pests can help detect and control pests in time. In this study, we built a lightweight agricultural pest identification method based on modified Yolov5s and reconstructed the original backbone network in tandem with MobileNetV3 to considerably reduce the number of parameters in the network model. At the same time, the ECA attention mechanism was introduced into the MobileNetV3 shallow network to meet the aim of effectively enhancing the network’s performance by introducing a limited number of parameters. A weighted bidirectional feature pyramid network (BiFPN) was utilized to replace the path aggregation network (PAnet) in the neck network to boost the feature extraction of tiny targets. The SIoU loss function was utilized to replace the CIoU loss function to increase the convergence speed and accuracy of the model prediction frame. The updated model was designated ECMB-Yolov5. In this study, we conducted experiments on eight types of common pest dataset photos, and comparative experiments were conducted using common target identification methods. The final model was implemented on an embedded device, the Jetson Nano, for real-time detection, which gave a reference for further application to UAV or unmanned cart real-time detection systems. The experimental results indicated that ECMB-Yolov5 decreased the number of parameters by 80.3% and mAP by 0.8% compared to the Yolov5s model. The real-time detection speed deployed on embedded devices reached 15.2 FPS, which was 5.7 FPS higher than the original model. mAP was improved by 7.1%, 7.3%, 9.9%, and 8.4% for ECMB-Yolov5 compared to Faster R-CNN, Yolov3, Yolov4, and Yolov4-tiny models, respectively. It was verified through experiments that the improved lightweight method in this study had a high detection accuracy while significantly reducing the number of parameters and accomplishing real-time detection.