3DeepM: An Ad Hoc Architecture Based on Deep Learning Methods for Multispectral Image Classification

Abstract

Current predefined architectures for deep learning are computationally very heavy and use tens of millions of parameters. Thus, computational costs may be prohibitive for many experimental or technological setups. We developed an ad hoc architecture for the classification of multispectral images using deep learning techniques. The architecture, called 3DeepM, is composed of 3D filter banks especially designed for the extraction of spatial-spectral features in multichannel images. The new architecture has been tested on a sample of 12210 multispectral images of seedless table grape varieties: Autumn Royal, Crimson Seedless, Itum4, Itum5 and Itum9. 3DeepM was able to classify 100% of the images and obtained the best overall results in terms of accuracy, number of classes, number of parameters and training time compared to similar work. In addition, this paper presents a flexible and reconfigurable computer vision system designed for the acquisition of multispectral images in the range of 400 nm to 1000 nm. The vision system enabled the creation of the first dataset consisting of 12210 37-channel multispectral images (12 VIS + 25 IR) of five seedless table grape varieties that have been used to validate the 3DeepM architecture. Compared to predefined classification architectures such as AlexNet, ResNet or ad hoc architectures with a very high number of parameters, 3DeepM shows the best classification performance despite using 130-fold fewer parameters than the architecture to which it was compared. 3DeepM can be used in a multitude of applications that use multispectral images, such as remote sensing or medical diagnosis. In addition, the small number of parameters of 3DeepM make it ideal for application in online classification systems aboard autonomous robots or unmanned vehicles.

A novel ground truth multispectral image dataset with weight, anthocyanins, and Brix index measures of grape berries tested for its utility in machine learning pipelines

“…The contrast and brightness do change the pixel values, but its range was limited to minimize any possible hindrance in the learning process of the classifiers. The precise values of the transformation ranges are identical to those described in the literature [ 33 ].…”

“…The 3D-CNN architecture employs two 3-dimensional convolutional blocks with convolutional layers that use dilated 3-dimensional kernels [ 33 ]. It also uses mean and average 3-dimensional pooling layers between these blocks and a total of 2 dense layers.…”

From Pixels to Deposits: Porphyry Mineralization With Multispectral Convolutional Neural Networks

“…One of the common approaches for multispectral image classification is to use 3D convolutional neural networks (CNNs) that can capture both spatial and spectral features from the input images. [30] proposed an ad hoc architecture called 3DeepM that consists of 3D filter banks specially designed to extract spatial spectral features in multichannel images. The authors applied their method to several aerial and satellite image datasets and achieved competitive results.…”

Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

“…MSI super-resolution through deep learning has become a promising method to recover additional spectral information from RGB images without using more expensive MSI or HSI hardware [6]. MSIs have been shown to be superior in classification tasks compared to corresponding RGB images [7,8]. Optimal waveband selection has been shown to improve the performances of model predictions [9,10].…”