Multi-Channel Multi-Modal Concatenation-based Deep Learning Model for Leaf Infection and Soil Property Prediction

Abstract

One of the most important tasks in improving crop yield quality is accurately predicting leaf infection and its related soil properties. A Multi-channel Convolutional Neural Network (MCNN) was already designed that utilized a separate channels for learning features of soil and leaf infection images. But, it was not able to capture the spatiotemporal variance of the leaf infections and may lose data because of feature fusion at the decision stage. Hence, this article proposes a Multi-channel Multimodal Concatenation-based CNN with Long Short-Term Memory (M 2 C 2 NN-LSTM) model to improve the generalizability of feature learning for leaf infection and soil property prediction. At first, the MCNN architecture is built to learn the deep features from soil and leaf images together using DenseNets followed by the Convolutional LSTM (ConvLSTM), which helps to extract the spatiotemporal dependencies between them. During feature learning, three different types of concatenation strategies are employed to fuse the encoding of spatiotemporal features with better generalization ability and achieve robust prediction. Once the prediction process is completed, the predicted outcomes of leaf infections and related soil properties are broadcasted to the cultivators via smartphones to develop yield productivity. At last, this model is validated by the different categories of leaf infection and soil photos for cotton, pineapple, and strawberry crops plants. The testing outcomes demonstrate that the M 2 C 2 NN-LSTM attains a mean accuracy of 90.3% for leaf infection prediction and 91.85% for soil property prediction than the conventional classifiers.