L*a*b*Fruits: A Rapid and Robust Outdoor Fruit Detection System Combining Bio-Inspired Features with One-Stage Deep Learning Networks

Abstract

Automation of agricultural processes requires systems that can accurately detect and classify produce in real industrial environments that include variation in fruit appearance due to illumination, occlusion, seasons, weather conditions, etc. In this paper we combine a visual processing approach inspired by colour-opponent theory in humans with recent advancements in one-stage deep learning networks to accurately, rapidly and robustly detect ripe soft fruits (strawberries) in real industrial settings and using standard (RGB) camera input. The resultant system was tested on an existent data-set captured in controlled conditions as well our new real-world data-set captured on a real strawberry farm over two months. We utilise F 1 score, the harmonic mean of precision and recall, to show our system matches the state-of-the-art detection accuracy ( F 1 : 0.793 vs. 0.799) in controlled conditions; has greater generalisation and robustness to variation of spatial parameters (camera viewpoint) in the real-world data-set ( F 1 : 0.744); and at a fraction of the computational cost allowing classification at almost 30fps. We propose that the L*a*b*Fruits system addresses some of the most pressing limitations of current fruit detection systems and is well-suited to application in areas such as yield forecasting and harvesting. Beyond the target application in agriculture this work also provides a proof-of-principle whereby increased performance is achieved through analysis of the domain data, capturing features at the input level rather than simply increasing model complexity.

The use of light spectrum blocking films to reduce populations of Drosophila suzukii Matsumura in fruit crops

“…It is therefore clear that the need to remain outdoors (expressed by a preference for UV light) is dominant in D. suzukii , but a secondary attraction to wavelengths (orange and red) that provide a possible secondary mechanism allowing for robust visual fruit detection was also noticed. Colour-opponency is thought to underpin colour perception in fruit flies, and has recently been used to enhance the performance of artificial fruit detectors 36 . Computational modelling of the sensory and neural perception systems facilitating fruit detection in insects will allow evermore targeted and nuanced interventions and provides an excellent channel for future investigation.…”

Fast and Accurate Detection of Banana Fruits in Complex Background Orchards

“…In the study of fruit detection, apple fruit detection and branch segmentation are the focus of researchers [30][31][32][33]; The establishment of a dedicated neural network for mango detection continues to emerge [34][35][36][37] ； Various neural networks in litchi [38,39], grape [40,41], strawberry [42,43] have achieved good results in their application. The detection of pomelo [44], kiwi fruit [45], waxberry [46], guava [47], and other fruits have been gradually concerned; With the development of deep learning, fruit flower detection, which is difficult to the traditional algorithm, has been emerging [48][49][50][51].…”

Analysis of the Pneumatic System Parameters of the Suction Cup Integrated with the Head for Harvesting Strawberry Fruit

“…Kirk et al [22] presented an analysis of the vision system used to identify the collected strawberries. Hayashi et al [15] presented a commercial robot model for harvesting strawberries.…”