3D Locating System for Pests’ Laser Control Based on Multi-Constraint Stereo Matching

Abstract

To achieve pest elimination on leaves with laser power, it is essential to locate the laser strike point on the pest accurately. In this paper, Pieris rapae (L.) (Lepidoptera: Pieridae), similar in color to the host plant, was taken as the object and the method for identifying and locating the target point was researched. A binocular camera unit with an optical filter of 850 nm wavelength was designed to capture the pest image. The segmentation of the pests’ pixel area was performed based on Mask R-CNN. The laser strike points were located by extracting the skeleton through an improved ZS thinning algorithm. To obtain the 3D coordinates of the target point precisely, a multi-constrained matching method was adopted on the stereo rectification images and the subpixel target points in the images on the left and right were optimally matched through fitting the optimal parallax value. As the results of the field test showed, the average precision of the ResNet50-based Mask R-CNN was 94.24%. The maximum errors in the X-axis, the Y-axis, and the Z-axis were 0.98, 0.68, and 1.16 mm, respectively, when the working depth ranged between 400 and 600 mm. The research was supposed to provide technical support for robotic pest control in vegetables.

Detection and Positioning of Camellia oleifera Fruit Based on LBP Image Texture Matching and Binocular Stereo Vision

“…The rectangular anchor frame regions of C. oleifera fruits can be obtained using the detection model to detect the fruits in the image, which are approximately the circumscribed rectangular frames of the contour of C. oleifera fruits. The anchor frame regions corresponding to the same C. oleifera fruit in the left and right images need to be matched through the multiple constraint conditions combining sequence consistency and epipolar constraint [28]. Considering that the anchor frames of the same C. oleifera fruit in the left and right images through the detection model may have deviation in the vertical direction, the matching range of the epipolar constraint principle is expanded to 5 pixel points above and below, as shown in Figure 2a.…”

Robots and Autonomous Machines for Sustainable Agriculture Production

“…The first category has eleven papers under the following sub-heading: Intelligent sensing for the crop or machine system [1,4,7,9,10,12,16,17,22,23,26]. Currently, a large number of studies focus on deep learning techniques, which have shown their superb impact on robotic sensing applications, as reflected in this issue.…”

“…Currently, a large number of studies focus on deep learning techniques, which have shown their superb impact on robotic sensing applications, as reflected in this issue. Some papers utilized improved YOLO-based [7,23,26], CNN-based [12,16] or RCNN-based [9] methods to developed a detection model for operating target recognition or performance evaluation from the RGB images. To achieve more accuracy, faster and compacter models may be popular due to the cost-effective and feasiblity with low-computing platforms.…”