Effect of fusing features from multiple DCNN architectures in image classification

Abstract

Automatic image classification has become a necessary task to handle the rapidly growing digital image usage. It has branched out many algorithms and adopted new techniques. Among them, feature fusion-based image classification methods rely on hand-crafted features traditionally. However, it has been proven that the bottleneck features extracted through pretrained convolutional neural networks (CNNs) can improve the classification accuracy. Thence, this study analyses the effect of fusing such cues from multiple architectures without being tied to any hand-crafted features. First, the CNN features are extracted from three different pre-trained models, namely AlexNet, VGG-16, and Inception-V3. Then, a generalised feature space is formed by employing principal component reconstruction and energy-level normalisation, where the features from individual CNN are mapped into a common subspace and embedded using arithmetic rules to construct fused feature vectors (FFVs). This transformation play a vital role in creating a representation that is appearance invariant by capturing complementary information of different high-level features. Finally, a multi-class linear support vector machine is trained. The experimental results demonstrate that such multi-modal CNN feature fusion is well suited for image/object classification tasks, but surprisingly it has not been explored so far by the computer vision research community extensively.

Classification of Marine Vessels with Multi-Feature Structure Fusion

“…In the Formulas (12) and (13), d(a, b) represents the Euclidean distance of vectors a and b, and t is selected to be 0.4. The weight matrix W 1 and W 2 reflect the relationship between each class center and others.…”

“…The three features consist of multi-scale completed local binary patterns (MS-CLBP), Bag of visual words (BOVW), and spatial pyramid matching (SPM). Methods based on CNN mainly refer to select the features of a certain layer in the convolutional neural network, and then use the support vector machine (SVM), extreme learning machine (ELM) or logistic regression classifier [12][13][14][15][16][17][18]. The literature [5] has used the 15th layer feature extracted from the pre-trained model vgg-16 based on the ImageNet dataset.…”

“…The literature [5] has used the 15th layer feature extracted from the pre-trained model vgg-16 based on the ImageNet dataset. Literature [12] has adopted AlexNet, VGG-16 and Inception-V3 to extract the features of the image, and then normalizes the different features into the same feature space. Literature [13] has designed a convolutional neural network extraction feature, and combined the traditional Gabor feature with MS-CLBP feature to describe the ship's target.…”

“…Considering the fact that a single feature may not be comprehensive enough for representing an image, some scholars have made further explorations on feature fusion. Some existing feature fusion methods mostly have used simple concatenation of different features in series [12,13], which do not consider the heterogeneous characteristics between different features. Sun et al [21] have adopted Canonical Correlation Analysis (CCA) for feature fusion to achieve compression of feature vector dimensions.…”

Dog Breed Identification with Fine tuning of Pre-trained models

“…Journal of Recent Technology and Engineering (IJRTE) ISSN: 2277-3878, Volume-8, Issue-2S11, September 2019output layer. It contains multiple 3X3 filters one after the other [5]. …”