Multi-level feature fusion for fruit bearing branch keypoint detection

Sun, Qixin; Chai, Xiujuan; Zeng, Zhikang; Zhou, Guomin; Sun, Tan

doi:10.1016/j.compag.2021.106479

Cited by 16 publications

(8 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Rong [12] proposed a tomato peduncle detection method for autonomous harvesting based on an improved you only look once (YOLO) method and RGB-D camera. Sun [13] proposed a multi-level feature fusion method for citrus bearing branch keypoint detection. In [14], Weyler proposed a leaf keypoint detection method to estimate the count of sugar beet leaves.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Vegetable Size Measurement Based on Stereo Camera and Keypoints Detection

Zheng

Sun

Meng

et al. 2022

Sensors

View full text Add to dashboard Cite

This work focuses on the problem of non-contact measurement for vegetables in agricultural automation. The application of computer vision in assisted agricultural production significantly improves work efficiency due to the rapid development of information technology and artificial intelligence. Based on object detection and stereo cameras, this paper proposes an intelligent method for vegetable recognition and size estimation. The method obtains colorful images and depth maps with a binocular stereo camera. Then detection networks classify four kinds of common vegetables (cucumber, eggplant, tomato and pepper) and locate six points for each object. Finally, the size of vegetables is calculated using the pixel position and depth of keypoints. Experimental results show that the proposed method can classify four kinds of common vegetables within 60 cm and accurately estimate their diameter and length. The work provides an innovative idea for solving the vegetable’s non-contact measurement problems and can promote the application of computer vision in agricultural automation.

show abstract

Section: Introductionmentioning

confidence: 99%

“…Keypoint detection is usually used to locate the stalk in automatic picking [12,13], guiding the robot to harvest fruits automatically. However, post-processing is still necessary to estimate the fruits' size.…”

Section: Introductionmentioning

confidence: 99%

Vegetable Size Measurement Based on Stereo Camera and Keypoints Detection

Zheng

Sun

Meng

et al. 2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Meanwhile, we have noticed that in recent years, keypoint detection algorithms have been widely applied in fruit picking location research due to their unique advantages in detecting stem-like target objects. And this has important implications for our research (20)(21)(22).…”

Section: Introductionmentioning

confidence: 99%

High-throughput plant phenotyping analysis of Melon (Cucumis melo L.) germplasm resources using deep learning methods and computer vision

Xu,

Shen,

Wei

et al. 2024

Preprint

View full text Add to dashboard Cite

Cucumis melo L., commonly known as melon, is a crucial horticultural crop. The selection and breeding of superior melon germplasm resources play a pivotal role in enhancing its marketability. However, current methods for melon appearance phenotypic analysis rely primarily on expert judgment and intricate manual measurements, which are not only inefficient but also costly. Therefore, to expedite the breeding process of melon, we analyzed the images of 117 melon varieties from two annual years utilizing artificial intelligence (AI) technology. By integrating the semantic segmentation model Dual Attention Network (DANet), the object detection model RTMDet, the keypoint detection model RTMPose, and the Mobile-Friendly Segment Anything Model (MobileSAM), a deep learning algorithm framework was constructed, capable of efficiently and accurately segmenting melon fruit and pedicel. On this basis, a series of feature extraction algorithms were designed, successfully obtaining 11 phenotypic traits of melon. Linear fitting verification results of selected traits demonstrated a high correlation between the algorithm-predicted values and manually measured true values, thereby validating the feasibility and accuracy of the algorithm. Moreover, cluster analysis using all traits revealed a high consistency between the classification results and genotypes. Finally, a user-friendly software was developed to achieve rapid and automatic acquisition of melon phenotypes, providing an efficient and robust tool for melon breeding, as well as facilitating in-depth research into the correlation between melon genotypes and phenotypes.

show abstract

“…They used YOLOv5 to detect the litchi main stem and used PSPNet to segment the main stem and obtained the pixel coordinates of picking points in the segmented image. Sun et al [25] proposed a novel method to detect the keypoint on the branch. They first obtained a candidate area according to the fruit-growing position and the fruit stem keypoint detection and used a multi-level feature fusion network to further detect the keypoint.…”

Section: Introductionmentioning

confidence: 99%

A Dragon Fruit Picking Detection Method Based on YOLOv7 and PSP-Ellipse

Zhou

Zhang

Wang

2023

Sensors

View full text Add to dashboard Cite

Dragon fruit is one of the most popular fruits in China and Southeast Asia. It, however, is mainly picked manually, imposing high labor intensity on farmers. The hard branches and complex postures of dragon fruit make it difficult to achieve automated picking. For picking dragon fruits with diverse postures, this paper proposes a new dragon fruit detection method, not only to identify and locate the dragon fruit, but also to detect the endpoints that are at the head and root of the dragon fruit, which can provide more visual information for the dragon fruit picking robot. First, YOLOv7 is used to locate and classify the dragon fruit. Then, we propose a PSP-Ellipse method to further detect the endpoints of the dragon fruit, including dragon fruit segmentation via PSPNet, endpoints positioning via an ellipse fitting algorithm and endpoints classification via ResNet. To test the proposed method, some experiments are conducted. In dragon fruit detection, the precision, recall and average precision of YOLOv7 are 0.844, 0.924 and 0.932, respectively. YOLOv7 also performs better compared with some other models. In dragon fruit segmentation, the segmentation performance of PSPNet on dragon fruit is better than some other commonly used semantic segmentation models, with the segmentation precision, recall and mean intersection over union being 0.959, 0.943 and 0.906, respectively. In endpoints detection, the distance error and angle error of endpoints positioning based on ellipse fitting are 39.8 pixels and 4.3°, and the classification accuracy of endpoints based on ResNet is 0.92. The proposed PSP-Ellipse method makes a great improvement compared with two kinds of keypoint regression method based on ResNet and UNet. Orchard picking experiments verified that the method proposed in this paper is effective. The detection method proposed in this paper not only promotes the progress of the automatic picking of dragon fruit, but it also provides a reference for other fruit detection.

show abstract

Multi-level feature fusion for fruit bearing branch keypoint detection

Cited by 16 publications

References 24 publications

Vegetable Size Measurement Based on Stereo Camera and Keypoints Detection

Vegetable Size Measurement Based on Stereo Camera and Keypoints Detection

High-throughput plant phenotyping analysis of Melon (Cucumis melo L.) germplasm resources using deep learning methods and computer vision

A Dragon Fruit Picking Detection Method Based on YOLOv7 and PSP-Ellipse

Contact Info

Product

Resources

About