Image classification with orchard metadata

Abstract

Low cost and easy to use monocular vision systems are able to capture large scale, dense data in orchards, to facilitate precision agriculture applications. Accurate image parsing is required for this purpose, however, operating in natural outdoor conditions makes this a complex task due to the undesirable intra-class variations caused by changes in illumination, pose and tree types, etc. Typically these variations are difficult to explicitly model and discriminative classifiers strive to be invariant to them. However, given the presence of structure, in both the orchard and how the data was obtained, a subset of these factors of variations can correlate with readily available metadata, including extrinsic experimental information such as the sun incidence angle, position within farm, etc. This paper presents a method to incorporate such metadata to aid scene parsing based on a multi-scale Multi-Layered Perceptron (MLP) architecture. Experimental results are shown for pixel segmentation over data collected at an apple orchard, leading to fruit detection and yield estimation. The results show a consistent improvement in segmentation accuracy with the inclusion of metadata under different network complexities, training configurations and evaluation metrics.

Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards

“…We subsequently provide an orchard block yield map, which can enable the grower to optimize their farm operations. This paper extends from our previous work with evaluation using new segmentation architectures, including different configurations of the previously used MLP architecture (Bargoti & Underwood, ) and the more widely used convolutional neural networks (CNNs). Additional developments are also made for the fruit detection algorithms and performance metrics, and an in‐depth discussion about the practical viability of the image processing approach is presented.…”

“…Where there is correlation between metadata and appearance variations and/or class distributions, including metadata can improve classification performance. Available at no extra cost to typical data capturing process, our previous works (Bargoti & Underwood, , ) have illustrated the use of metadata in allowing simpler classifiers to capture that space and provide a performance boost, leading to similar performance with reduced training exemplars.…”

“…Additional developments are also made for the fruit detection algorithms and performance metrics, and an in‐depth discussion about the practical viability of the image processing approach is presented. The primary contributions from this paper are: Image fruit segmentation analysis using the previously benchmarked multiscale multilayered perceptron (MLP; Bargoti & Underwood, ; Hung et al., ) and more recent convolutional neural networks (CNN). A study of the utility of different metadata sources and their inclusion within the different classification architectures and training configurations. Analysis of the impact of accurate image segmentation toward fruit detection and yield estimation.…”

Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry

“…Methods include a variety of platforms, such as manned and unmanned ground vehicles (UGVs) [4,5,6,7,8], unmanned aerial vehicles (UAVs) and hand-held sensors [9]. Different types of imaging sensors have also been used, including “standard” (visible light) cameras and stereo, near infra-red, long wave thermal infrared cameras and LiDAR [2,3,4,9,10,11,12,13,14].…”

“…In the latter case, a process of calibration to manual field-counts can be done, which has proven to be accurate for some canopy types, including trellised apple orchards [5,6,15], almond orchards [7] and vineyards [18,19]. The calibration process requires manual field or harvest counts, which is labour intensive and would ideally be repeated every year.…”