Experiments and Analysis of Close-Shot Identification of On-Branch Citrus Fruit with RealSense

Liu, Jizhan; Yan, Yuan; Zhou, Yao; Zhu, Xinxin; Syed, Tabinda Naz

doi:10.3390/s18051510

Cited by 21 publications

(9 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the case of unpruned trees, the lack of correlation between real and modelled fruits can be due to an important shading problem [52,53]. Even if occlusion is a natural phenomenon in trees [23], produced by leaves or fruit clusters [13], the higher leaf density of the unpruned trees importantly accentuates this problem.…”

Section: Orange Count With K-means Algorithmmentioning

confidence: 99%

In-Field Estimation of Orange Number and Size by 3D Laser Scanning

et al. 2019

View full text Add to dashboard Cite

The estimation of fruit load of an orchard prior to harvest is useful for planning harvest logistics and trading decisions. The manual fruit counting and the determination of the harvesting capacity of the field results are expensive and time-consuming. The automatic counting of fruits and their geometry characterization with 3D LiDAR models can be an interesting alternative. Field research has been conducted in the province of Cordoba (Southern Spain) on 24 'Salustiana' variety orange trees-Citrus sinensis (L.) Osbeck-(12 were pruned and 12 unpruned). Harvest size and the number of each fruit were registered. Likewise, the unitary weight of the fruits and their diameter were determined (N = 160). The orange trees were also modelled with 3D LiDAR with colour capture for their subsequent segmentation and fruit detection by using a K-means algorithm. In the case of pruned trees, a significant regression was obtained between the real and modelled fruit number (R 2 = 0.63, p = 0.01). The opposite case occurred in the unpruned ones (p = 0.18) due to a leaf occlusion problem. The mean diameters proportioned by the algorithm (72.15 ± 22.62 mm) did not present significant differences (p = 0.35) with the ones measured on fruits (72.68 ± 5.728 mm). Even though the use of 3D LiDAR scans is time-consuming, the harvest size estimation obtained in this research is very accurate.In the literature, diverse works of detection in different types of fruits or harvest can be found, such as almond [2], apple [3][4][5][6][7], cherryfruit [8], cucumber [9], mango [10,11], orange [12,13], pineapple [14,15], or tomato [16].Fruit detection requires segmentation, shape selection, and identification phases [17]. Segmentation consists of filtering through a colour threshold of the components of the scene that can be considered fruit [18]. Different characteristics of perimeter, area, or compaction allow to select the shapes (blobs) and to identify the fruits one by one.In bidimensional models, circles can be detected by the Hough transform [19] or by adjusting circular contours [20]. The colour cameras allowed Harrell et al. [12] to implement a robotic orange harvesting system and Grasso and Recce [21] to perform RGB segmentation. Qureshi et al.[11] used clustering of K-nearest neighbour pixels. Qureshi et al. [22] presented a texture-based method for shape recognition using an over-segmentation of super-pixels from the local gradient calculation.Colour segmentation using charge coupled device cameras returns pixels with RGB graduations or colour composition by addition of the primary colours red, green, and blue, allowing rapid detection of ripe fruit [23]. However, its drawback is its false positives. In addition, shape detection has a high computational cost. Therefore, a suitable alternative is the use of a colour filter followed by shape detection to avoid these false positives.The colour and shape characteristics allow us to approach its count using 2D photos, filtering by colour or chromaticity, delimiting shapes by contour, and being...

show abstract

Section: Orange Count With K-means Algorithmmentioning

confidence: 99%

In-Field Estimation of Orange Number and Size by 3D Laser Scanning

et al. 2019

View full text Add to dashboard Cite

show abstract

“…However, the image obtained at a long distance has too few peduncle pixels, which makes it difficult to use the algorithm to segment peduncle of tomato bunches. If the camera is close to the peduncle to collect images, the larger pixel area of the peduncle is easier to be segmented, and a better point cloud data for the peduncle can be obtained by RGB-D camera [19]. Besides, close-up images can also avoid background interference to a certain extent.…”

Section: Methodsmentioning

confidence: 99%

A peduncle detection method of tomato for autonomous harvesting

Rong

Dai

Wang

2021

Complex Intell. Syst.

View full text Add to dashboard Cite

For automating the harvesting of bunches of tomatoes in a greenhouse, the end-effector needs to reach the exact cutting point and adaptively adjust the pose of peduncles. In this paper, a method is proposed for peduncle cutting point localization and pose estimation. Images captured in real time at a fixed long-distance are detected using the YOLOv4-Tiny detector with a precision of 92.7% and a detection speed of 0.0091 s per frame, then the YOLACT + + Network with mAP of 73.1 and a time speed of 0.109 s per frame is used to segment the close-up distance. The segmented peduncle mask is fitted to the curve using least squares and three key points on the curve are found. Finally, a geometric model is established to estimate the pose of the peduncle with an average error of 4.98° in yaw angle and 4.75° in pitch angle over the 30 sets of tests.

show abstract

“…Machine vision technologies are widely employed for the detection, location of ripe fruits (Liu et al, 2018; Luo et al, 2018a; Xiong et al, 2018) and yield prediction (Qureshi et al, 2017). Taking pictures at night can well avoid sunlight and environmental interference.…”

Section: Related Workmentioning

confidence: 99%

Design, development, and field evaluation of a rubber tapping robot

Zhou

Zhang

et al. 2021

Journal of Field Robotics

View full text Add to dashboard Cite

Modern robotics has made great strides in the field of agricultural harvesting. In the field of rubber tapping, however, there are still significant challenges to be faced. As one of the few agricultural harvesting areas primarily manual, rubber tapping requires precise control of the knife in the forest's unstructured environment at night. This paper presents the design of a rubber cutting robot for natural rubber plantations. An upgraded spatial spiral trajectory is established by analyzing the trunk and manual rubber cutting trajectories for the operation of a six‐axis tandem robot arm. A compact binocular stereo vision system achieved the initial perception of the parameters required for the cutting trajectory. Besides, an integrated end‐effector was developed to enable further precise perception of the cutting trajectory and cutting operations. The system was tested in the field at a rubber plantation in Hainan Province. The upgraded trajectory's adaptability and the accuracy and efficiency of the starting point positioning algorithm were verified. The operational performance of the complete system was evaluated to help guide future improvements. The error of the start point locating is 1.0 ± 0.1 mm, and the execution time is 17.01 ± 3.65 s. The bark consumption—cutting depth setting of (2.0 and 5.0 mm) is more appropriate for the rubber tapping robot. Then the accuracy of chip thickness and chip width is about 1.73 ± 0.28 mm, and 5.07 ± 0.13 mm, respectively. Furthermore, the accuracy of chip weight is 1.99 ± 0.24 mm. The average operating time of the whole tapping operation is 80 ± 5 s. Future work will include improvements in system stiffness and robustness. This study demonstrates the potential of applying industrial robotics to the field of latex harvesting. Robotics may be a future development towards mechanized rubber tapping.

show abstract

Experiments and Analysis of Close-Shot Identification of On-Branch Citrus Fruit with RealSense

Cited by 21 publications

References 26 publications

In-Field Estimation of Orange Number and Size by 3D Laser Scanning

In-Field Estimation of Orange Number and Size by 3D Laser Scanning

A peduncle detection method of tomato for autonomous harvesting

Design, development, and field evaluation of a rubber tapping robot

Contact Info

Product

Resources

About