MMM and CLCFM: A 3D point cloud reconstruction method based on photogrammetry for all-around images taken by rotating an individual plant

Abstract

This research proposes a novel technique for acquiring a large amount of high-density, high-precision 3D point cloud data for plants. We propose two methods, multi-masked matching (MMM) and the closed-loop coarse-to-fine method (CLCFM). The proposed approach addresses challenges in reconstructing plant 3D point clouds from all-around images using SfM and multi-view stereo methods. Given the complex structure of plants, with thin objects like leaves and stems overlapping, reconstructing accurate 3D point clouds presents significant challenges. One of these is occlusion, where points in the 3D point cloud cannot be obtained due to overlapping parts, preventing accurate point capture. Another is the generation of erroneous points in non-existent locations due to image matching errors along object outlines. The MMM mitigates these problems through an iterative process that alternates between locally generating point clouds and refining these point clouds by removing erroneous points using mask images. The proposed pipeline enables the construction of plant 3D point clouds with minimal noise and fewer missing points. By facilitating the acquisition of high-density, high-precision 3D data on a large number of plant bodies, as required for research activities, the proposed method is expected to enable comparative analysis of visible phenotypes in the growth process of a wide range of plant species based on 3D information.