Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN

Hu, Heming; Kaizu, Yutaka; Zhang, Hongduo; Xu, Yang; Imou, Kenji; Li, Ming; Huang, Jingjing; Dai, Sihui

doi:10.25165/j.ijabe.20221506.7306

Cited by 8 publications

(4 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting 3D models also contain richer visual information. In contrast, satellite remote sensing methods have lower resolution, poor real-time performance, and lower accuracy in elevation information [23][24][25] . Moreover, using drones with oblique photography mode can more accurately measure the height and position information of crops and obstacles in the field.…”

Section: Resultsmentioning

confidence: 99%

Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Zhao,

Wang,

Wen

et al. 2023

International Journal of Agricultural and Biological Engineering

View full text Add to dashboard Cite

Digital twins can improve the level of control over physical entities and help manage complex systems by integrating a range of technologies. The autonomous agricultural machine has shown revolutionary effects on labor reduction and utilization rate in field works. Autonomous vehicles in precision agriculture have the potential to improve competitiveness compared to current crop production methods and have become a research hotspot. However, the development time and resources required in experiments have limited the research in this area. Simulation tools in unmanned farming that are required to enable more efficient, reliable, and safe autonomy are increasingly demanding. Inspired by the recent development of an open-source virtual simulation platform, this study proposed an autoware-based simulator to evaluate the performance of agricultural machine guidance based on digital twins. Oblique photogrammetry using drones is used to construct threedimensional maps of fields at the same scale as reality. A communication format suitable for agricultural machines was developed for data input and output, along with an inter-node communication methodology. The conversion, publishing, and maintenance of multiple coordinate systems were completed based on ROS (Robot Operating System). Coverage path planning was performed using hybrid curves based on Bézier curves, and it was tested in both a simulation environment and actual fields with the aid of Pure Pursuit algorithms and PID controllers.

show abstract

Section: Resultsmentioning

confidence: 99%

Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Zhao,

Wang,

Wen

et al. 2023

International Journal of Agricultural and Biological Engineering

View full text Add to dashboard Cite

show abstract

“…Furthermore, these methods can be classified based on the scale of data acquisition, encompassing population-scale detection and individual plant-scale data detection. These methods leverage streamlined procedures to analyze the acquired phenotype information, facilitating subsequent qualitative or quantitative analysis [14][15][16][17][18] .…”

Section: Introductionmentioning

confidence: 99%

Method for the real-time detection of tomato ripeness using a phenotype robot and RP-YolactEdge

Yuanqiao,

Wenbo,

Chuanyu

et al. 2024

International Journal of Agricultural and Biological Engineering

View full text Add to dashboard Cite

In order to address the challenge of non-destructive detection of tomato fruit ripeness in controlled environments, this study proposed a real-time instance segmentation method based on the edge device. This method combined the principles of phenotype robots and machine vision based on deep learning. A compact and remotely controllable phenotype detection robot was employed to acquire precise data on tomato ripeness. The video data were then processed by using an efficient backbone and the FeatFlowNet structure for feature extraction and analysis of key-frame to non-key-frame mapping from video data. To enhance the diversity of training datasets and the generalization of the model, an innovative approach was chosen by using random enhancement techniques. Besides, the PolyLoss optimization technique was applied to further improve the accuracy of the ripeness multi-class detection tasks. Through validation, the method of this study achieved real-time processing speeds of 90.1 fps (RTX 3070Ti) and 65.5 fps (RTX 2060 S), with an average detection accuracy of 97% compared to manually measured results. This is more accurate and efficient than other instance segmentation models according to actual testing in a greenhouse. Therefore, the results of this research can be deployed in edge devices and provide technical support for unmanned greenhouse monitoring devices or fruit-picking robots in facility environments.

show abstract

“…Therefore, one-stage detection algorithms can achieve real-time target detection, but compared to two-stage detection algorithms, they may have slightly lower accuracy. Heming Hu et al achieved high accuracy in strawberry detection by combining two-stage detection (Mask R-CNN) and one-stage detection (YOLOv3) networks for training and recognition [15]. To solve multiple complex situations in actual traffic scenes, Yalin Miao et al proposed a deep learning target detection network based on SI-SSD [16].…”

Section: Introductionmentioning

confidence: 99%

SGST-YOLOv8: An Improved Lightweight YOLOv8 for Real-Time Target Detection for Campus Surveillance

Cheng,

Chao,

Yang

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Real-time target detection plays an important role in campus intelligent surveillance systems. This paper introduces Soft-NMS, GSConv, Triplet Attention, and other advanced technologies to propose a lightweight pedestrian and vehicle detection model named SGST-YOLOv8. In this paper, the improved YOLOv8 model is trained on the self-made dataset, and the tracking algorithm is combined to achieve an accurate and efficient real-time pedestrian and vehicle tracking detection system. The improved model achieved an accuracy of 88.6%, which is 1.2% higher than the baseline model YOLOv8. Additionally, the mAP0.5:0.95 increased by 3.2%. The model parameters and GFLOPS reduced by 5.6% and 7.9%, respectively. In addition, this study also employed the improved YOLOv8 model combined with the bot sort tracking algorithm on the website for actual detection. The results showed that the improved model achieves higher FPS than the baseline YOLOv8 model when detecting the same scenes, with an average increase of 3–5 frames per second. The above results verify the effectiveness of the improved model for real-time target detection in complex environments.

show abstract

Recognition and localization of strawberries from 3D binocular cameras for a strawberry picking robot using coupled YOLO/Mask R-CNN

Cited by 8 publications

References 21 publications

Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Digital twins in smart farming: An autoware-based simulator for autonomous agricultural vehicles

Method for the real-time detection of tomato ripeness using a phenotype robot and RP-YolactEdge

SGST-YOLOv8: An Improved Lightweight YOLOv8 for Real-Time Target Detection for Campus Surveillance

Contact Info

Product

Resources

About