Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion

Gong

et al. 2020

Sensors

134

Deep Learning (DL) is the state-of-the-art machine learning technology, which shows superior performance in computer vision, bioinformatics, natural language processing, and other areas. Especially as a modern image processing technology, DL has been successfully applied in various tasks, such as object detection, semantic segmentation, and scene analysis. However, with the increase of dense scenes in reality, due to severe occlusions, and small size of objects, the analysis of dense scenes becomes particularly challenging. To overcome these problems, DL recently has been increasingly applied to dense scenes and has begun to be used in dense agricultural scenes. The purpose of this review is to explore the applications of DL for dense scenes analysis in agriculture. In order to better elaborate the topic, we first describe the types of dense scenes in agriculture, as well as the challenges. Next, we introduce various popular deep neural networks used in these dense scenes. Then, the applications of these structures in various agricultural tasks are comprehensively introduced in this review, including recognition and classification, detection, counting and yield estimation. Finally, the surveyed DL applications, limitations and the future work for analysis of dense images in agriculture are summarized.

Section: Counting and Yield Estimationmentioning

confidence: 99%

“…Besides, in dense agricultural scenes, for background processing with high similarity to the object, using semantic segmentation to extract the region of interest can achieve good results [66,78].…”

Section: (4) Solutions To Challengesmentioning

confidence: 99%

Applications of Deep Learning for Dense Scenes Analysis in Agriculture: A Review

Zhang

Gong

et al. 2020

Sensors

134

“…Similar to our previous work [4], we use a combination of the Kanade-Lucas-Tomasi (KLT) optical flow estimator, Kalman Filter, and Hungarian Assignment algorithm to track fruits across image frames. However, we improve upon previous work by defining a different filtering step which fuses both the Faster R-CNN detections and KLT estimates as measurements.…”

Section: A Tracking In the Image Planementioning

confidence: 99%

Monocular Camera Based Fruit Counting and Mapping With Semantic Data Association

Chen

IEEE Robot. Autom. Lett.

et al. 2019

Self Cite

We present a cheap, lightweight, and fast fruit counting pipeline. Our pipeline relies only on a monocular camera, and achieves counting performance comparable to a state-of-the-art fruit counting system that utilizes an expensive sensor suite including a monocular camera, LiDAR and GPS/INS on a mango dataset. Our pipeline begins with a fruit and tree trunk detection component that uses state-of-the-art convolutional neural networks (CNNs). It then tracks fruits and tree trunks across images, with a Kalman Filter fusing measurements from the CNN detectors and an optical flow estimator. Finally, fruit count and map are estimated by an efficient fruit-as-feature semantic structure from motion (SfM) algorithm which converts 2D tracks of fruits and trunks into 3D landmarks, and uses these landmarks to identify double counting scenarios. There are many benefits of developing such a low cost and lightweight fruit counting system, including applicability to agriculture in developing countries, where monetary constraints or unstructured environments necessitate cheaper hardware solutions.Index Terms-Robotics in agriculture and forestry, deep learning in robotics and automation, visual tracking, mapping, object detection, segmentation and categorization.

“…Another classification model based on a convolutional neural network that is capable of extracting the features automatically with an accuracy of 97.19% [5]. A fully convolutional network (FCN) is trained and segmented the image in two categories: one in the fruit pixels and other in non-fruit pixels [6]. The convolutional neural network-based an effective model is developed for the classification of various Durio Zibethinus.…”

Section: Previous Workmentioning

confidence: 99%

Automatic Detection and Classification of Nutrients Deficiency in Fruit Based on Automated Machine Learning

Yogesh*,

Dubey,

Ratan

2019

IJEAT

Machine learning-based classification and detection of surface defect of fruit involve manual feature identification and selection from input datasets. Deep learning discovers the useful features from the input data. This approach simplifies the training of the neural network and makes them faster. The selection of useful patterns from the fruit features results in better accuracy. The number of layers represents the depth of the model. Neural network provides learning to the model. As the dataset contains many features. It is obvious that all features are not relevant to the system. The proposed system learns from these features by identifying the pattern and select the relevant features. This is the most crucial phase of the machine learning to identify the appropriate features to make the system faster and accurate. In this paper, we propose solving fruit surface defect detection using Automated Machine Learning (AML). The outcome is the prediction of the fruit surface defect in terms of probability due to nutrient deficiency