On-Tree Mango Fruit Size Estimation Using RGB-D Images

Abstract

In-field mango fruit sizing is useful for estimation of fruit maturation and size distribution, informing the decision to harvest, harvest resourcing (e.g., tray insert sizes), and marketing. In-field machine vision imaging has been used for fruit count, but assessment of fruit size from images also requires estimation of camera-to-fruit distance. Low cost examples of three technologies for assessment of camera to fruit distance were assessed: a RGB-D (depth) camera, a stereo vision camera and a Time of Flight (ToF) laser rangefinder. The RGB-D camera was recommended on cost and performance, although it functioned poorly in direct sunlight. The RGB-D camera was calibrated, and depth information matched to the RGB image. To detect fruit, a cascade detection with histogram of oriented gradients (HOG) feature was used, then Otsu’s method, followed by color thresholding was applied in the CIE L*a*b* color space to remove background objects (leaves, branches etc.). A one-dimensional (1D) filter was developed to remove the fruit pedicles, and an ellipse fitting method employed to identify well-separated fruit. Finally, fruit lineal dimensions were calculated using the RGB-D depth information, fruit image size and the thin lens formula. A Root Mean Square Error (RMSE) = 4.9 and 4.3 mm was achieved for estimated fruit length and width, respectively, relative to manual measurement, for which repeated human measures were characterized by a standard deviation of 1.2 mm. In conclusion, the RGB-D method for rapid in-field mango fruit size estimation is practical in terms of cost and ease of use, but cannot be used in direct intense sunshine. We believe this work represents the first practical implementation of machine vision fruit sizing in field, with practicality gauged in terms of cost and simplicity of operation.

Non-Contact Body Measurement for Qinchuan Cattle with LiDAR Sensor

“…The point cloud data (PCD) acquired by a LiDAR sensor can reflect the 3D mapping of research targets or automated environmental monitoring data. Regarding the applications of the Kinect RGB-D camera, it is widely applied and convenient for direct proximal data acquisition of research targets, such as the estimation of fruit sizes of on-tree mangoes [ 7 ], 3D measurements of maize plant height [ 8 ] and automated behavior recognition of pigs [ 9 ]. For the planar scanning applications of SICK LMS series sensors, larger range of view sight data could be obtained, such as 3D plant reconstruction of maize [ 10 ], fruit yield mapping of individual trees in almond orchards [ 11 ] and leaf area index estimation in vineyards [ 12 ].…”

Calibration of In-Plane Center Alignment Errors in the Installation of a Circular Slide with Machine-Vision Sensor and a Reflective Marker

“…Prior to calibrating the IPCA using the previously selected machine-vision camera, as shown in Figure 10 , camera calibration should be performed. Here, error factors are identified and calibrated, such as the distortion of the lens (including the radial distortion and tangential distortion that arise when a point in a three-dimensional space is mapped onto a two-dimensional image plane) and installation uncertainties [ 25 , 26 , 27 ].…”

Fruit Morphological Measurement Based on Three-Dimensional Reconstruction

“…Qureshi et al [28] proposed two new methods for automated counting of fruit in images of mango tree canopies, one using texture-based dense segmentation and one using shape-based fruit detection, then compared the use of these methods relative to existing techniques. Wang et al [29] developed a one-dimensional filter to remove the fruit pedicles and employed an ellipse fitting method to identify well-separated fruit. Stein et al [30] presented a novel multi-sensor framework using a multiple viewpoint approach to solve the problem of occlusion to efficiently identify, track, localize, and map every piece of fruit in a commercial mango orchard.…”