Fast 3D Rotation Estimation of Fruits Using Spheroid Models

Abstract

Automated fruit inspection using cameras involves the analysis of a collection of views of the same fruit obtained by rotating a fruit while it is transported. Conventionally, each view is analyzed independently. However, in order to get a global score of the fruit quality, it is necessary to match the defects between adjacent views to prevent counting them more than once and assert that the whole surface has been examined. To accomplish this goal, this paper estimates the 3D rotation undergone by the fruit using a single camera. A 3D model of the fruit geometry is needed to estimate the rotation. This paper proposes to model the fruit shape as a 3D spheroid. The spheroid size and pose in each view is estimated from the silhouettes of all views. Once the geometric model has been fitted, a single 3D rotation for each view transition is estimated. Once all rotations have been estimated, it is possible to use them to propagate defects to neighbor views or to even build a topographic map of the whole fruit surface, thus opening the possibility to analyze a single image (the map) instead of a collection of individual views. A large effort was made to make this method as fast as possible. Execution times are under 0.5 ms to estimate each 3D rotation on a standard I7 CPU using a single core.

Single Fusion Image from Collections of Fruit Views for Defect Detection and Classification

“…For this purpose, we followed the method described in our previous work [ 15 ]. Several conditions must be fulfilled for this method to work correctly: The fruit must exhibit some texture so that rotation is observable in the images.…”

“…Let be the view’s 2D image coordinates of the point of interest (relative to the fruit’s center of mass). We can estimate the height of this point, , using the geometrical model as described in [ 15 ]. The absolute fruit coordinates are obtained by projecting vector onto the fruit unit vectors at the current view: …”

“…This paper can be considered an extension of our previous work [ 15 ] about how to estimate the 3D rotation between consecutive views of fruit by assuming a spheroidal model. Rotation estimation from the views is crucial since the assumption of slip-less rotation fails in the case of real fruit.…”

“…Rotation estimation from the views is crucial since the assumption of slip-less rotation fails in the case of real fruit. The central idea in [ 15 ] was to estimate the rotation between consecutive views in order to predict the position of a point of one view in the next one. On the other hand, the work presented in this paper relies on those rotation estimations in order to synthesize a single image which represents the whole surface of the fruit.…”

Advancing Mixed Reality Digital Twins Through 3D Reconstruction of Fresh Produce

“…Xie et al [20] employed 3D PCD (Point Cloud Data) reconstruction to model the geometric shape of Rosa roxburghii fruit. Albiol et al [21] estimated 3D fruit rotation using spheroid models, enabling defect analysis and surface examination. Prawiranto et al [22] developed a 3D continuum model to study apple fruit dehydration at the cellular scale, considering changes in cellular structure and their impact on sorption isotherm and water permeability.…”