3D shape sensing and deep learning-based segmentation of strawberries

Louedec, Justin Le; Cielniak, Grzegorz

doi:10.1016/j.compag.2021.106374

Cited by 21 publications

(7 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such conditions lend themselves to 3D representation, as multiple unoccluded viewpoints are possible, allowing for a full representation of the fruit (Feldmann & Tabb, 2022; He et al., 2017; Le Louëdec & Cielniak, 2020; Li, Cockerton et al., 2020). To be practically useful to breeding and selection, however, shape needs to be assessed in real agricultural environments, for which there is potential although further research into this is needed (Le Louëdec & Cielniak, 2021a).…”

Section: Automation Of Morphological Traits Currently Used In Breedingmentioning

confidence: 99%

“…However, CNNs are reasonably robust to variance in illumination, and this can be further addressed through methods such as merging features from different colour spaces (Kirk et al., 2020). The unstructured, complex environment also poses challenges in terms of occlusion of the organs under evaluation by other fruit, flowers, stems or leaves, and cluttered backgrounds make segmentation difficult (Fan et al., 2022; Kirk et al., 2020; Lamb & Chuah, 2018; Lin & Chen, 2018; Yu et al., 2019; Zhou et al., 2020), but imaging from multiple viewpoints (Kerfs et al., 2017) and 3D sensing have the potential to assist with this (Le Louëdec & Cielniak, 2021a). Fruit characteristics, such as the small size of the fruit and variation in appearance, have also been noted as further obstacles in agricultural settings (Fan et al., 2022; Kirk et al., 2020), along with sensor‐related restrictions, such as available camera viewpoints, low contrast, variance in both colour balance and saturation and the interference of the sun on infra‐red based sensors (Heylen et al., 2021; Kirk et al., 2020; Le Louëdec & Cielniak, 2021a).…”

Section: High‐throughput Image‐based Phenotypingmentioning

confidence: 99%

See 1 more Smart Citation

High‐throughput phenotyping for breeding targets—Current status and future directions of strawberry trait automation

James

Whitehouse

Cielniak

2022

Plants People Planet

Self Cite

View full text Add to dashboard Cite

Strawberry breeders are faced with increasing demands by propagators, growers, retailers and consumers for particular agronomic traits. This and the volume of plants requiring assessment during selection constrain breeders to rapid and qualitative rating methods. High-throughput systems for assessing these traits automatically could indicate which families, or individual genotypes, should be singled out for further, more thorough evaluation, thus significantly increasing the selection intensity and accuracy. This review assesses the current status of and future potential for automated phenotyping in strawberry crops, highlighting key advances and the gaps which need to be addressed to facilitate the development of such technology.

show abstract

Section: Automation Of Morphological Traits Currently Used In Breedingmentioning

confidence: 99%

Section: High‐throughput Image‐based Phenotypingmentioning

confidence: 99%

High‐throughput phenotyping for breeding targets—Current status and future directions of strawberry trait automation

James

Whitehouse

Cielniak

2022

Plants People Planet

Self Cite

View full text Add to dashboard Cite

show abstract

“…With the advancement of new information technology and the promotion of technical methods, machine learning (ML) and deep learning (DL) have made significant strides in scene recognition and object classification. Considering their characteristics of faster detection, better generalization, and stronger robustness, these methods have also emerged as a research hotspot in strawberry detection and recognition ( Yu et al., 2019 ; Pérez-Borrero et al., 2020 ; Le Louëdec and Cielniak, 2021 ). The current strawberry ripeness detection method predominantly revolve around the integration of ML, DL, and hyperspectral imaging techniques.…”

Section: Introductionmentioning

confidence: 99%

A fine recognition method of strawberry ripeness combining Mask R-CNN and region segmentation

Tang

Wang

et al. 2023

Front. Plant Sci.

View full text Add to dashboard Cite

As a fruit with high economic value, strawberry has a short ripeness period, and harvesting at an incorrect time will seriously affect the quality of strawberries, thereby reducing economic benefits. Therefore, the timing of its harvesting is very demanding. A fine ripeness recognition can provide more accurate crop information, and guide strawberry harvest management more timely and effectively. This study proposes a fine recognition method for field strawberry ripeness that combines deep learning and image processing. The method is divided into three stages: In the first stage, self-calibrated convolutions are added to the Mask R-CNN backbone network to improve the model performance, and then the model is used to extract the strawberry target in the image. In the second stage, the strawberry target is divided into four sub-regions by region segmentation method, and the color feature values of B, G, L, a and S channels are extracted for each sub-region. In the third stage, the strawberry ripeness is classified according to the color feature values and the results are visualized. Experimental results show that with the incorporation of self-calibrated convolutions into the Mask R-CNN, the model’s performance has been substantially enhanced, leading to increased robustness against diverse occlusion interferences. As a result, the final average precision (AP) has improved to 0.937, representing a significant increase of 0.039 compared to the previous version. The strawberry ripeness classification effect is the best on the SVM classifier, and the accuracy under the combined channel BGLaS reaches 0.866. The classification results are better than common manual feature extraction methods and AlexNet, ResNet18 models. In order to clarify the role of the region segmentation method, the contribution of different sub-regions to each ripeness is also explored. The comprehensive results demonstrate that the proposed method enables the evaluation of six distinct ripeness levels of strawberries in the complex field environment. This method can provide accurate decision support for strawberry refined planting management.

show abstract

“…These methods learn to segment plant parts based on a set of training examples ( van Dijk et al., 2020 ). Louëdec and Cielniak (2021) , for example, used the position and surface normals of the 3D points as input, to learn to segment strawberries among canopies using an encoder-decoder convolutional neural network (CNN). Boogaard et al.…”

Section: Introductionmentioning

confidence: 99%

3D data-augmentation methods for semantic segmentation of tomato plant parts

et al. 2023

View full text Add to dashboard Cite

Introduction3D semantic segmentation of plant point clouds is an important step towards automatic plant phenotyping and crop modeling. Since traditional hand-designed methods for point-cloud processing face challenges in generalisation, current methods are based on deep neural network that learn to perform the 3D segmentation based on training data. However, these methods require a large annotated training set to perform well. Especially for 3D semantic segmentation, the collection of training data is highly labour intensitive and time consuming. Data augmentation has been shown to improve training on small training sets. However, it is unclear which data-augmentation methods are effective for 3D plant-part segmentation.MethodsIn the proposed work, five novel data-augmentation methods (global cropping, brightness adjustment, leaf translation, leaf rotation, and leaf crossover) were proposed and compared to five existing methods (online down sampling, global jittering, global scaling, global rotation, and global translation). The methods were applied to PointNet++ for 3D semantic segmentation of the point clouds of three cultivars of tomato plants (Merlice, Brioso, and Gardener Delight). The point clouds were segmented into soil base, stick, stemwork, and other bio-structures.Results and disccusionAmong the data augmentation methods being proposed in this paper, leaf crossover indicated the most promising result which outperformed the existing ones. Leaf rotation (around Z axis), leaf translation, and cropping also performed well on the 3D tomato plant point clouds, which outperformed most of the existing work apart from global jittering. The proposed 3D data augmentation approaches significantly improve the overfitting caused by the limited training data. The improved plant-part segmentation further enables a more accurate reconstruction of the plant architecture.

show abstract

3D shape sensing and deep learning-based segmentation of strawberries

Cited by 21 publications

References 45 publications

High‐throughput phenotyping for breeding targets—Current status and future directions of strawberry trait automation

High‐throughput phenotyping for breeding targets—Current status and future directions of strawberry trait automation

A fine recognition method of strawberry ripeness combining Mask R-CNN and region segmentation

3D data-augmentation methods for semantic segmentation of tomato plant parts

Contact Info

Product

Resources

About