Localization and Classification of Rice-grain Images Using Region Proposals-based Convolutional Neural Network

Abstract

This paper proposes a solution to localization and classification of rice grains in an image. All existing related works rely on conventional based machine learning approaches. However, those techniques do not do well for the problem designed in this paper, due to the high similarities between different types of rice grains. The deep learning based solution is developed in the proposed solution. It contains pre-processing steps of data annotation using the watershed algorithm, auto-alignment using the major axis orientation, and image enhancement using the contrast-limited adaptive histogram equalization (CLAHE) technique. Then, the mask region-based convolutional neural networks (R-CNN) is trained to localize and classify rice grains in an input image. The performance is enhanced by using the transfer learning and the dropout regularization for overfitting prevention. The proposed method is validated using many scenarios of experiments, reported in the forms of mean average precision (mAP) and a confusion matrix. It achieves above 80% mAP for main scenarios in the experiments. It is also shown to perform outstanding, when compared to human experts.

PhosopNet: An improved grain localization and classification by image augmentation

“…Not only classification but also localization was necessary for grain quality inspection. As the highlight, Mask R-CNN [50] with ResNet [46] was used for grain localization and classification (called MIMR [29]). But a large number of manual labeling on too many small grains [51] was still necassary.…”

“…To do more with less data, this paper named PhosopNet proposes the image augmentation that generates the thoundsand grain data from hundred one, instead of manually labeling those ten-thoundsand small grain images. For the expansion of previous works, the computer vision applied to rice or grain problems (both bag of words [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23] and CNN [24][25][26][27][28][29]) can achieve high performance by training the less labeled grain data.…”

“…Since the previous papers on grain (or rice-grain) recognition (in both bag of words [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23] and convolutional neural network [24][25][26][27][28][29]) require a large number of human-annotated labels (labeled data), the proposed PhosopNet leverages the image augmentation by feature transformation to artificially generate a variety of grain instances as a larger dataset and train them to the model. Practically, a grain object that is performed by image processing operations: rotation, brightness adjustment and horizontal flipping, in order to increase a number of data, as shown in Figure 2.…”

“…The proposed PhosopNet was compared to previous highlight paper MIMR [29] that was based on ResNet-50 [46] for classification. Since the PhosopNet classification was DenseNet that also had image augmentation to increase the number of grains in training set, instead of the full labeling by human.…”

“…Many researchers leveraged computer vision as an inspection solution since the beginning of "Japanese rice grading problem" in 2002 [7]. From the literature, all papers could be categorized by methods [8] into 2 groups: bag of words [9][10][11][12][13][14][15][16][17][18][19][20][21][22][23] and convolutional neural network (CNN) [24][25][26][27][28][29]. The former was used by early researches [30] (which require less number of labeled data [31]) that were still useful in some open-world industry [20][21][22] such as iRSVPred [23].…”