Multi-Feature Fusion with Convolutional Neural Network for Ship Classification in Optical Images

2022

Remote Sensing

The mainstream algorithms used for ship classification and detection can be improved based on convolutional neural networks (CNNs). By analyzing the characteristics of ship images, we found that the difficulty in ship image classification lies in distinguishing ships with similar hull structures but different equipment and superstructures. To extract features such as ship superstructures, this paper introduces transformer architecture with self-attention into ship classification and detection, and a CNN and Swin transformer model (CNN-Swin model) is proposed for ship image classification and detection. The main contributions of this study are as follows: (1) The proposed approach pays attention to different scale features in ship image classification and detection, introduces a transformer architecture with self-attention into ship classification and detection for the first time, and uses a parallel network of a CNN and a transformer to extract features of images. (2) To exploit the CNN’s performance and avoid overfitting as much as possible, a multi-branch CNN-Block is designed and used to construct a CNN backbone with simplicity and accessibility to extract features. (3) The performance of the CNN-Swin model is validated on the open FGSC-23 dataset and a dataset containing typical military ship categories based on open-source images. The results show that the model achieved accuracies of 90.9% and 91.9% for the FGSC-23 dataset and the military ship dataset, respectively, outperforming the existing nine state-of-the-art approaches. (4) The good extraction effect on the ship features of the CNN-Swin model is validated as the backbone of the three state-of-the-art detection methods on the open datasets HRSC2016 and FAIR1M. The results show the great potential of the CNN-Swin backbone with self-attention in ship detection.

Section: Introductionmentioning

confidence: 99%

Fine-Grained Ship Classification by Combining CNN and Swin Transformer

Huang

2022

Remote Sensing

“…Based on the pixel-level fusion of visible and infrared bispectral images, Gao ( 2020 ) integrated the algorithms that include image preprocessing, image smoothing, and anti-cloud interference to achieve the detection of ship targets in complex land and sea backgrounds. Ren et al ( 2019 ) proposed a CNN framework with fewer layers and parameter. This method used the Softmax function to classify different ship types and achieved good results.…”

Section: Introductionmentioning

confidence: 99%

Recognition and Classification of Ship Images Based on SMS-PCNN Model

Huang

et al. 2022

Front. Neurorobot.

In the field of ship image recognition and classification, traditional algorithms lack attention to the differences between the grain of ship images. The differences in the hull structure of different categories of ships are reflected in the coarse-grain, whereas the differences in the ship equipment and superstructures of different ships of the same category are reflected in the fine-grain. To extract the ship features of different scales, the multi-scale paralleling CNN oriented on ships images (SMS-PCNN) model is proposed in this paper. This model has three characteristics. (1) Extracting image features of different sizes by parallelizing convolutional branches with different receptive fields. (2) The number of channels of the model is adjusted two times to extract features and eliminate redundant information. (3) The residual connection network is used to extend the network depth and mitigate the gradient disappearance. In this paper, we collected open-source images on the Internet to form an experimental dataset and conduct performance tests. The results show that the SMS-PCNN model proposed in this paper achieves 84.79% accuracy on the dataset, which is better than the existing four state-of-the-art approaches. By the ablation experiments, the effectiveness of the optimization tricks used in the model is verified.

“…The application of deep learning in image classification, recognition, and tracking has yielded positive results [ 29 , 30 , 31 , 32 ]. Deep learning builds network models by imitating the neural network of the human brain and using efficient learning strategies to obtain results through multilevel analyses and calculations.…”

Section: Introductionmentioning

confidence: 99%

Low-Illumination Image Enhancement in the Space Environment Based on the DC-WGAN Algorithm

Jiang

et al. 2021

Sensors

Owing to insufficient illumination of the space station, the image information collected by the intelligent robot will be degraded, and it will not be able to accurately identify the tools required for the robot’s on-orbit maintenance. This situation increases the difficulty of the robot’s maintenance in a low-illumination environment. We proposes a novel enhancement method for images under low-illumination, namely, a deep learning algorithm based on the combination of deep convolutional and Wasserstein generative adversarial networks (DC-WGAN) in CIELAB color space. The original low-illuminance image is converted from the RGB space to the CIELAB color space which is relatively close to human vision, to accurately estimate the illumination image, and effectively reduce the effect of uneven illumination. DC-WGAN is applied to enhance the brightness component by increasing the width of the generation network to obtain more image features. Subsequently, the LAB is converted into RGB space to obtain the final enhanced image. The feasibility of the algorithm is verified by experiments on low-illuminance image under general, special, and actual conditions and comparing the experimental results with four commonly used algorithms. This study lays a technical foundation for robot target recognition and on-orbit maintenance in a space environment.