Improved Point-Cloud Segmentation for Plant Phenotyping Through Class-Dependent Sampling of Training Data to Battle Class Imbalance

Abstract

Plant scientists and breeders require high-quality phenotypic data. However, obtaining accurate manual measurements for large plant populations is often infeasible, due to the high labour requirement involved. This is especially the case for more complex plant traits, like the traits defining the plant architecture. Computer-vision methods can help in solving this bottleneck. The current work focusses on methods using 3D point cloud data to obtain phenotypic datasets of traits related to the plant architecture. A first step is the segmentation of the point clouds into plant organs. One of the issues in point-cloud segmentation is that not all plant parts are equally represented in the data and that the segmentation performance is typically lower for minority classes than for majority classes. To address this class-imbalance problem, we used a common practice to divide large point clouds into chunks that were independently segmented and recombined later. In our case, the chunks were created by selecting anchor points and combining those with points in their neighbourhood. As a baseline, the anchor points were selected in a class-independent way, representing the class distribution in the original data. Then, we propose a class-dependent sampling strategy to battle class imbalance. The difference in segmentation performance between the class-independent and the class-dependent training set was analysed first. Additionally, the effect of the number of points selected as the neighbourhood was investigated. Smaller neighbourhoods resulted in a higher level of class balance, but also in a loss of context that was contained in the points around the anchor point. The overall segmentation quality, measured as the mean intersection-over-union (IoU), increased from 0.94 to 0.96 when the class-dependent training set was used. The biggest class improvement was found for the “node,” for which the percentage of correctly segmented points increased by 46.0 percentage points. The results of the second experiment clearly showed that higher levels of class balance did not necessarily lead to better segmentation performance. Instead, the optimal neighbourhood size differed per class. In conclusion, it was demonstrated that our class-dependent sampling strategy led to an improved point-cloud segmentation method for plant phenotyping.

A graph-based approach for simultaneous semantic and instance segmentation of plant 3D point clouds

“…Using only geometry-based features is not enough to accurately segment a point cloud, as it is for example hard to discriminate petioles from branches ( Boogaard et al., 2022 ). Our proposal to include botanical knowledge encoded as a semantic quotient tree into the segmentation process enables us to control the consistency of the segmentation and refine it if necessary, both semantically (e.g., deciding between petioles and branches) and per instance (e.g., to split a petiole from the associated leaf blade).…”

“…Unfortunately, in the case of plants, most methods only allow two types of organs: stem and leaf blade ( Ziamtsov and Navlakha, 2019 ; Liu et al., 2021 ; Li et al., 2022a ; Li et al., 2022b ; Li et al., 2022c ), sometimes adding the ground when it has not been removed beforehand ( Schunck et al., 2021 ). Interestingly ( Boogaard et al., 2022 ), proposes a method which distinguishes between stem and petioles, even adding the labels “growing point” (similar to “apex” in our case), “node”, “ovary” and “tendril”. Noticing that leaf blades typically contain more points than the other organs, this method uses the state-of-the-art PointNet++ architecture ( Qi et al., 2017 ) and adds a strategy to counter the effects of such class imbalance.…”

3D data-augmentation methods for semantic segmentation of tomato plant parts

“…Some automatic high-throughput plant phenotyping systems are used, but these are often based on 2D images, which limit the accuracy of the estimation of geometric traits, such as internode diameter, and leaf area (Boogaard et al, 2020). With the rapid development of 3D sensors including 3D scanners, LiDARs and RGB-D cameras, more methods become available for 3D plant phenotyping, e.g., Johann et al (2015); Golbach et al (2016); Itakura and Hosoi (2018); Magistri et al (2020); Boogaard et al (2022). This paper has a focus on methods for 3D segmentation of plant parts as an important prerequisite to extract geometrical plant traits (Boogaard et al, 2021).…”

Digital plant phenotyping in three dimensions: what’s the point?

“…In this work, we applied the classdependent sampling strategy, using the default chunk size of 4,096 points. (Boogaard et al, 2022).…”

“…The colours represent the classes as specified in the legend, the black squares show a zoomed in segment of the point cloud. Adapted from(Boogaard et al, 2022).…”