Pruning Points Detection of Sweet Pepper Plants Using 3D Point Clouds and Semantic Segmentation Neural Network

Giang, Truong Thi Huong; Ryoo, Young-Jae

doi:10.3390/s23084040

Cited by 5 publications

(8 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It uses the ORB and BA algorithms to extract feature points and optimize camera poses in local or global maps. Due to its advantages, we proposed a method to create 3D point clouds by using ORB-SLAM3 and then optimizing the result with the help of the Iterative Closest Point (ICP) method [ 21 ]. The process is described in Figure 2 .…”

Section: Related Workmentioning

confidence: 99%

“…We then located the pruning regions by generating 3D semantic point clouds. The center point of a pruning region is considered a pruning position [ 21 ]. This method has some drawbacks.…”

Section: Proposed Autonomous Robotic Systemmentioning

confidence: 99%

“…The system is complex and comprises several modules. Our previous research has focused on semantic segmentation neural networks to recognize plant parts and how to build 3D semantic point clouds based on sequences of RGB-D images [ 20 , 21 ]. In this paper, we present a method to find the pruning position and direction from the 3D semantic point clouds, as well as the controlling robot arm program and an end-to-end autonomous pruning system.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Autonomous Robotic System to Prune Sweet Pepper Leaves Using Semantic Segmentation with Deep Learning and Articulated Manipulator

Giang,

Ryoo

2024

Biomimetics

Self Cite

View full text Add to dashboard Cite

This paper proposes an autonomous robotic system to prune sweet pepper leaves using semantic segmentation with deep learning and an articulated manipulator. This system involves three main tasks: the perception of crop parts, the detection of pruning position, and the control of the articulated manipulator. A semantic segmentation neural network is employed to recognize the different parts of the sweet pepper plant, which is then used to create 3D point clouds for detecting the pruning position and the manipulator pose. Eventually, a manipulator robot is controlled to prune the crop part. This article provides a detailed description of the three tasks involved in building the sweet pepper pruning system and how to integrate them. In the experiments, we used a robot arm to manipulate the pruning leaf actions within a certain height range and a depth camera to obtain 3D point clouds. The control program was developed in different modules using various programming languages running on the ROS (Robot Operating System).

show abstract

Section: Related Workmentioning

confidence: 99%

“…We then located the pruning regions by generating 3D semantic point clouds. The center point of a pruning region is considered a pruning position [ 21 ]. This method has some drawbacks.…”

Section: Proposed Autonomous Robotic Systemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Autonomous Robotic System to Prune Sweet Pepper Leaves Using Semantic Segmentation with Deep Learning and Articulated Manipulator

Giang,

Ryoo

2024

Biomimetics

Self Cite

View full text Add to dashboard Cite

show abstract

“…Giang and Ryoo (2023) introduced an approach using semantic segmentation neural networks (NN), the ICP algorithm and ORB‐SLAM3, a visual SLAM application with a LiDAR camera, to create 3D‐point clouds of sweet peppers. An improved structure from motion (SFM) and patch‐based multiview stereo (PMVS) method based on similar graph clustering and graph matching are proposed by Zhang et al (2023) to perform 3D sparse and dense reconstruction of green plums by utilizing computer vision technology.…”

Section: Introductionmentioning

confidence: 99%

Implementing deep‐learning techniques for accurate fruit disease identification

Sujatha

Mahalakshmi

Chatterjee³

2023

Plant Pathology

View full text Add to dashboard Cite

To overcome the problems of manual identification of fruit disease, this work proposes a deep‐learning model to analyse fruit images to detect diseases in the fruit. We are proposing here a convolutional neural network (CNN)‐based model for fruit disease classification. By including many layers, the proposed CNN model extracts numerous features from the fruit, deals with the large data set and finally evaluates it. With the MobileNetv2 model, the disease prediction accuracy for papaya, guava and citrus was 99.4%, 98.8% and 95.8% and the recall values were 99.4%, 98.8% and 93.8%, respectively. With VGG16, the disease prediction accuracy for papaya, guava and citrus was 97.7%, 99.6% and 94.2% and the recall values were 96.5%, 99.6% and 89.2%, respectively. Finally, with DenseNet121, the disease prediction accuracy for papaya, guava and citrus was 99.4%, 97.6% and 99.2%, and the recall values were 98.8%, 97.6% and 99.2%, respectively.

show abstract

“…Utilizing machine vision techniques to acquire image-based 3D models of seedlings offers several advantages. It eliminates the need for strict research environments, mitigates costly expenses, and enables the capture of color, texture information, and improved measurement precision [26,27]. Sun et al [28] proposed a high-throughput, three-dimensional rapid plant point cloud reconstruction method based on autonomous calibration of the Kinect v2 sensor position.…”

Section: Introductionmentioning

confidence: 99%

Research on Morphological Indicator Extraction Method of Pinus massoniana Lamb. Based on 3D Reconstruction

Li,

Xia,

Liu

et al. 2023

Forests

View full text Add to dashboard Cite

Pinus massoniana (Lamb.) is an important plantation species in southern China. Accurate measurement of P. massoniana seedling morphological indicators is crucial for accelerating seedling quality assessment. Machine vision, with its objectivity and stability, can replace human eyes in performing these measurements. In this paper, a measurement method for seedling morphological indicators based on Euclidean distance, Laplacian contraction, PointNet++, and 3D reconstruction is proposed. Firstly, multi-angle sequence images of 30 one-year-old P. massoniana seedlings were collected, distorted, and corrected to generate a sparse point cloud through the Structure-from-Motion (SFM) and dense point cloud through the Patch-Based Multiple View Stereo (PMVS). Secondly, a Dense Weighted Semantic Segmentation Model based on PointNet++ was designed, achieving effective segmentation of the P. massoniana seedling point clouds. Finally, a multi-iteration plane method based on Laplacian contraction was proposed. The new skeleton points were refined by minimizing the Euclidean distance, iteratively generating the optimal morphological skeleton, thus facilitating the extraction of morphological indicators. The experimental results demonstrated a good correlation between the machine vision-extracted morphological indicators (including plant height, ground diameter, and height-to-diameter ratio) and manually measured data. The improved PointNet++ model achieved an accuracy of 0.9448 on the training set. The accuracy and Mean Intersection over Union (MIoU) of the test set reached 0.9430 and 0.7872, respectively. These findings can provide reliable technical references for the accurate assessment of P. massoniana seedling quality and the promotion of digital forestry construction.

show abstract

Pruning Points Detection of Sweet Pepper Plants Using 3D Point Clouds and Semantic Segmentation Neural Network

Cited by 5 publications

References 25 publications

Autonomous Robotic System to Prune Sweet Pepper Leaves Using Semantic Segmentation with Deep Learning and Articulated Manipulator

Autonomous Robotic System to Prune Sweet Pepper Leaves Using Semantic Segmentation with Deep Learning and Articulated Manipulator

Implementing deep‐learning techniques for accurate fruit disease identification

Research on Morphological Indicator Extraction Method of Pinus massoniana Lamb. Based on 3D Reconstruction

Contact Info

Product

Resources

About