Temporal Upsampling of Point Cloud Sequences by Optimal Transport for Plant Growth Visualization

Abstract

Plant growth visualization from a series of 3D scanner measurements is a challenging task. Time intervals between successive measurements are typically too large to allow a smooth animation of the growth process. Therefore, obtaining a smooth animation of the plant growth process requires a temporal upsampling of the point cloud sequence in order to obtain approximations of the intermediate states between successive measurements. Additionally, there are suddenly arising structural changes due to the occurrence of new plant parts such as new branches or leaves. We present a novel method that addresses these challenges via semantic segmentation and the generation of a segment hierarchy per scan, the matching of the hierarchical representations of successive scans and the segment‐wise computation of optimal transport. The transport problems' solutions yield the information required for a realistic temporal upsampling, which is generated in real time. Thereby, our method does not require shape templates, good correspondences or huge databases of examples. Newly grown and decayed parts of the plant are detected as unmatched segments and are handled by identifying corresponding bifurcation points and introducing virtual segments in the previous, respectively successive time step. Our method allows the generation of realistic upsampled growth animations with moderate computational effort.

Multi-scale Space-time Registration of Growing Plants

“…We have not carried a quantitative comparison since it is hard to design a proper evaluation metrics for the interpolation problem. Our framework achieves visually similar results to [15] with similar running time. A series of videos showing the interpolated point clouds with our approach can also be found in the supplementary material.…”

“…This seems particularly adapted to the case of plants because of their shape. Some methods dedicated to plants have been proposed in recent years [15,10]. They rely on plant specific properties, for example a semantic segmentation, to first establish shape correspondence then compute an optimal transformation.…”

“…They rely on plant specific properties, for example a semantic segmentation, to first establish shape correspondence then compute an optimal transformation. We also use a topological skeleton in our approach and compare the results with [15,10].…”

“…Points in P 2 without corresponding point in P 1 (the new emerging segments) can be attached to the matching points of their nearest neighbour. The set {p t } is the interpolated point cloud at t. Our results using this approach on a maize plant can be visually compared with the interpolation result of [15] visually, see the supplementary material. We have not carried a quantitative comparison since it is hard to design a proper evaluation metrics for the interpolation problem.…”

Topology-Change-Aware Volumetric Fusion for Dynamic Scene Reconstruction

“…There is also a set of works related to dynamic scene reconstruction but not focused on voxel-based techniques: 1) Other template/mesh-based deformation approaches [40,21,5]; 2) Methods for learning-based schemes that may handle larger changes [1,12,21,14,39,13]; 3) Methods on point correspondence based interpolation that do not require the prior of a mesh representation and are more flexible with respect to topological changes [23,41,44,2]; 4) Finally, some point distribution based approaches that do not require correspondence search and provide even more flexibility [8,35,17].…”